[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:44 dist-test-slave-0956 krb5kdc[11218](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:44 dist-test-slave-0956 krb5kdc[11218](info): set up 2 sockets
Jun 18 22:26:44 dist-test-slave-0956 krb5kdc[11218](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260618 22:26:45.431787 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.973s user 0.003s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:26:45 dist-test-slave-0956 krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:26:45.597833 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:35687
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:36315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:35687
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:45.932358 11231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:45.932675 11231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:45.932762 11231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:45.940574 11231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:45.940712 11231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:45.940758 11231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:45.940843 11231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:45.940899 11231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:45.951049 11231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:36315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:35687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:35687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:26:45.952409 11231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:45.953693 11231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:45.966619 11237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:45.966749 11236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:45.966526 11239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:45.967284 11231 server_base.cc:1034] running on GCE node
I20260618 22:26:45.968175 11231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:45.969866 11231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:45.971096 11231 hybrid_clock.cc:648] HybridClock initialized: now 1781821605971038 us; error 58 us; skew 500 ppm
I20260618 22:26:45.971462 11231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:26:45 dist-test-slave-0956 krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821605, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:45.979540 11231 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:26:45.981698 11231 webserver.cc:460] Webserver started at http://127.10.242.254:43613/ using document root <none> and password file <none>
I20260618 22:26:45.982564 11231 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:45.982705 11231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:45.983028 11231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:45.986258 11231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "8a61898fe0174298a0953ef714743b46"
format_stamp: "Formatted at 2026-06-18 22:26:45 on dist-test-slave-0956"
server_key: "d7b6ed017bbf353b19ec7595bc2e1032"
server_key_iv: "164f17e3e5757205989ddbd971afe824"
server_key_version: "encryptionkey@0"
I20260618 22:26:45.987066 11231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "8a61898fe0174298a0953ef714743b46"
format_stamp: "Formatted at 2026-06-18 22:26:45 on dist-test-slave-0956"
server_key: "d7b6ed017bbf353b19ec7595bc2e1032"
server_key_iv: "164f17e3e5757205989ddbd971afe824"
server_key_version: "encryptionkey@0"
I20260618 22:26:45.992813 11231 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260618 22:26:45.996805 11246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:45.998692 11231 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260618 22:26:45.998907 11231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "8a61898fe0174298a0953ef714743b46"
format_stamp: "Formatted at 2026-06-18 22:26:45 on dist-test-slave-0956"
server_key: "d7b6ed017bbf353b19ec7595bc2e1032"
server_key_iv: "164f17e3e5757205989ddbd971afe824"
server_key_version: "encryptionkey@0"
I20260618 22:26:45.999132 11231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:46.021826 11231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:46.026726 11231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:46.027016 11231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:46.046016 11231 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:35687
I20260618 22:26:46.046042 11297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:35687 every 8 connection(s)
I20260618 22:26:46.047392 11231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:46.051862 11298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:46.054529 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 11231
I20260618 22:26:46.054838 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:26:46.057262 11211 external_mini_cluster.cc:1428] Setting key fd9cc72b51951f1133c65fbf96043a18
I20260618 22:26:46.065189 11298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46: Bootstrap starting.
I20260618 22:26:46.068017 11298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:46.068961 11298 log.cc:824] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:46.071609 11298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46: No bootstrap required, opened a new log
I20260618 22:26:46.079509 11298 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a61898fe0174298a0953ef714743b46" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35687 } }
I20260618 22:26:46.079905 11298 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:46.080000 11298 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a61898fe0174298a0953ef714743b46, State: Initialized, Role: FOLLOWER
I20260618 22:26:46.080649 11298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a61898fe0174298a0953ef714743b46" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35687 } }
I20260618 22:26:46.080919 11298 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:46.081051 11298 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:46.081300 11298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:46.083607 11298 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a61898fe0174298a0953ef714743b46" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35687 } }
I20260618 22:26:46.084121 11298 leader_election.cc:302] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a61898fe0174298a0953ef714743b46; no voters:
I20260618 22:26:46.084754 11298 leader_election.cc:288] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:46.084860 11303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:46.086508 11303 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46 [term 1 LEADER]: Becoming Leader. State: Replica: 8a61898fe0174298a0953ef714743b46, State: Running, Role: LEADER
Jun 18 22:26:46 dist-test-slave-0956 krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:26:46.087164 11298 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:46.087113 11303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a61898fe0174298a0953ef714743b46" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35687 } }
I20260618 22:26:46.090354 11305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a61898fe0174298a0953ef714743b46. Latest consensus state: current_term: 1 leader_uuid: "8a61898fe0174298a0953ef714743b46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a61898fe0174298a0953ef714743b46" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35687 } } }
I20260618 22:26:46.090785 11305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:46.093396 11304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a61898fe0174298a0953ef714743b46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a61898fe0174298a0953ef714743b46" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35687 } } }
I20260618 22:26:46.093837 11304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:46.095090 11310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:46.101707 11310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:46.102659 11301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:46.061963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59364 (local address 127.10.242.254:35687)
0618 22:26:46.063975 (+ 2012us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:46.063989 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:46.064050 (+ 61us) server_negotiation.cc:407] Connection header received
0618 22:26:46.073391 (+ 9341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:46.073415 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:46.073824 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:46.074261 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:46.077198 (+ 2937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:46.078549 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:46.080434 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:46.081330 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:46.081522 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:46.088333 (+ 6811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:46.088345 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:46.088368 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:46.088432 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:46.094421 (+ 5989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:46.095734 (+ 1313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:46.095744 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:46.095757 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:46.096022 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:46.096744 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:46.096750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:46.096753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:46.097811 (+ 1058us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:46.098109 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:46.099543 (+ 1434us) server_negotiation.cc:299] Negotiation successful
0618 22:26:46.100042 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1778,"thread_start_us":1519,"threads_started":1}
I20260618 22:26:46.120280 11310 catalog_manager.cc:1347] Generated new cluster ID: 86e30462fa5a402f99a49fda9866551a
I20260618 22:26:46.120437 11310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:46.137506 11310 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:46.138875 11310 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:46.152060 11310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a61898fe0174298a0953ef714743b46: Generated new TSK 0
I20260618 22:26:46.153195 11310 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
I20260618 22:26:46.200899 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:35687
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:36315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:46.545135 11324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:46.545444 11324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:46.545583 11324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:46.552991 11324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:46.553156 11324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:46.553319 11324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:26:46.563946 11324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:36315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:35687
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:26:46.565358 11324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:46.566771 11324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:46.580608 11330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:46.580662 11332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:46.580632 11329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:46.581818 11324 server_base.cc:1034] running on GCE node
I20260618 22:26:46.582546 11324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:46.583549 11324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:46.584827 11324 hybrid_clock.cc:648] HybridClock initialized: now 1781821606584763 us; error 48 us; skew 500 ppm
I20260618 22:26:46.585218 11324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:26:46 dist-test-slave-0956 krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821606, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:46.588488 11324 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:26:46.590204 11324 webserver.cc:460] Webserver started at http://127.10.242.193:36477/ using document root <none> and password file <none>
I20260618 22:26:46.590909 11324 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:46.591030 11324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:46.591346 11324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:46.594473 11324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "c80c0e1f57714242a9b0d5c7ba7506a0"
format_stamp: "Formatted at 2026-06-18 22:26:46 on dist-test-slave-0956"
server_key: "f3062515b771be493a55d79f8cdb4e34"
server_key_iv: "4fb3a5e6daae91e458ca791be33971ea"
server_key_version: "encryptionkey@0"
I20260618 22:26:46.595345 11324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "c80c0e1f57714242a9b0d5c7ba7506a0"
format_stamp: "Formatted at 2026-06-18 22:26:46 on dist-test-slave-0956"
server_key: "f3062515b771be493a55d79f8cdb4e34"
server_key_iv: "4fb3a5e6daae91e458ca791be33971ea"
server_key_version: "encryptionkey@0"
I20260618 22:26:46.601624 11324 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260618 22:26:46.605751 11339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:46.607365 11324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:26:46.607580 11324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "c80c0e1f57714242a9b0d5c7ba7506a0"
format_stamp: "Formatted at 2026-06-18 22:26:46 on dist-test-slave-0956"
server_key: "f3062515b771be493a55d79f8cdb4e34"
server_key_iv: "4fb3a5e6daae91e458ca791be33971ea"
server_key_version: "encryptionkey@0"
I20260618 22:26:46.607811 11324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:46.641599 11324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:46.645951 11324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:46.646220 11324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:46.647187 11324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:46.648566 11324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:46.648645 11324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:46.648767 11324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:46.648818 11324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:46.682283 11324 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:41639
I20260618 22:26:46.682334 11451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:41639 every 8 connection(s)
I20260618 22:26:46.683964 11324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:26:46.687734 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 11324
I20260618 22:26:46.687963 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:26:46.688683 11211 external_mini_cluster.cc:1428] Setting key d92c0f3f9d5b9463107ffdb5a6f1641e
Jun 18 22:26:46 dist-test-slave-0956 krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821606, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.242.194@KRBTEST.COM; defaulting to no policy
I20260618 22:26:46.706136 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:46.686602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:46985 (local address 127.10.242.254:35687)
0618 22:26:46.687044 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:46.687049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:46.688218 (+ 1169us) server_negotiation.cc:407] Connection header received
0618 22:26:46.690364 (+ 2146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:46.690368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:46.690439 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:46.690536 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:46.692199 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:46.693205 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:46.694839 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:46.695435 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:46.695569 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:46.699702 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:46.699708 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:46.699711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:46.699747 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:46.702885 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:46.703906 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:46.703910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:46.703912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:46.704033 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:46.704532 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:46.704571 (+ 39us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:46.704573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:46.704838 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:46.704943 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:46.705719 (+ 776us) server_negotiation.cc:299] Negotiation successful
0618 22:26:46.705894 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":221,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.194@KRBTEST.COM" created.
I20260618 22:26:46.707283 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:46.687209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35687 (local address 127.10.242.193:46985)
0618 22:26:46.688025 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:46.688083 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:46.690067 (+ 1984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:46.690864 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:46.690873 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:46.691407 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:46.692000 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:46.692017 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:46.693360 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:46.693365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:46.694701 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:46.694714 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:46.695726 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:46.695733 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:46.696017 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:46.696705 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:46.696739 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:46.699415 (+ 2676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:46.703061 (+ 3646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:46.703072 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:46.703093 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:46.703774 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:46.704140 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:46.704146 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:46.704150 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:46.704423 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:46.704955 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:46.704962 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:46.705391 (+ 429us) client_negotiation.cc:769] Sending connection context
0618 22:26:46.705674 (+ 283us) client_negotiation.cc:240] Negotiation successful
0618 22:26:46.706110 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":372,"threads_started":1}
I20260618 22:26:46.708977 11452 heartbeater.cc:347] Connected to a master server at 127.10.242.254:35687
I20260618 22:26:46.709337 11452 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:46.710098 11452 heartbeater.cc:511] Master 127.10.242.254:35687 requested a full tablet report, sending...
I20260618 22:26:46.712599 11263 ts_manager.cc:194] Registered new tserver with Master: c80c0e1f57714242a9b0d5c7ba7506a0 (127.10.242.193:41639)
I20260618 22:26:46.714150 11263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:46985
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
I20260618 22:26:46.728761 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:35687
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:36315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:47.064810 11458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:47.065141 11458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:47.065271 11458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:47.072480 11458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:47.072597 11458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:47.072774 11458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:26:47.083307 11458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:36315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:35687
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:26:47.084676 11458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:47.086066 11458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:47.099151 11463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:47.099205 11466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:47.099064 11464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:47.100128 11458 server_base.cc:1034] running on GCE node
I20260618 22:26:47.100752 11458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:47.101737 11458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:47.102960 11458 hybrid_clock.cc:648] HybridClock initialized: now 1781821607102903 us; error 41 us; skew 500 ppm
I20260618 22:26:47.103317 11458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:26:47 dist-test-slave-0956 krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821607, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:47.106972 11458 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:26:47.108507 11458 webserver.cc:460] Webserver started at http://127.10.242.194:46879/ using document root <none> and password file <none>
I20260618 22:26:47.109227 11458 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:47.109362 11458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:47.109678 11458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:47.112681 11458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "07d7efa6d4f24acab9947419828866dc"
format_stamp: "Formatted at 2026-06-18 22:26:47 on dist-test-slave-0956"
server_key: "6e38f87dd6ce063df653308fb59ba495"
server_key_iv: "333364bef8831b22e0b49462d5b0bf10"
server_key_version: "encryptionkey@0"
I20260618 22:26:47.113489 11458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "07d7efa6d4f24acab9947419828866dc"
format_stamp: "Formatted at 2026-06-18 22:26:47 on dist-test-slave-0956"
server_key: "6e38f87dd6ce063df653308fb59ba495"
server_key_iv: "333364bef8831b22e0b49462d5b0bf10"
server_key_version: "encryptionkey@0"
I20260618 22:26:47.118780 11458 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:26:47.122334 11473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:47.123790 11458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:26:47.123952 11458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "07d7efa6d4f24acab9947419828866dc"
format_stamp: "Formatted at 2026-06-18 22:26:47 on dist-test-slave-0956"
server_key: "6e38f87dd6ce063df653308fb59ba495"
server_key_iv: "333364bef8831b22e0b49462d5b0bf10"
server_key_version: "encryptionkey@0"
I20260618 22:26:47.124153 11458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:47.148998 11458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:47.153767 11458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:47.154039 11458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:47.155055 11458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:47.156364 11458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:47.156441 11458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:47.156550 11458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:47.156600 11458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:47.188491 11458 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:44695
I20260618 22:26:47.188537 11585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:44695 every 8 connection(s)
I20260618 22:26:47.189781 11458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:26:47.195957 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 11458
I20260618 22:26:47.196134 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:26:47.196590 11211 external_mini_cluster.cc:1428] Setting key 4412d257fce42c17dc791aa59fb18ebf
Jun 18 22:26:47 dist-test-slave-0956 krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821607, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.242.195@KRBTEST.COM; defaulting to no policy
I20260618 22:26:47.210137 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:47.192306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:43609 (local address 127.10.242.254:35687)
0618 22:26:47.192519 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:47.192525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:47.193765 (+ 1240us) server_negotiation.cc:407] Connection header received
0618 22:26:47.196222 (+ 2457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:47.196227 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:47.196318 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:47.196407 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:47.198109 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:47.198878 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:47.200374 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:47.200801 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:47.200941 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:47.204518 (+ 3577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:47.204523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:47.204526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:47.204559 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:47.206959 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:47.207675 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:47.207680 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:47.207682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:47.207838 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:47.208395 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:47.208398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:47.208401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:47.208667 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:47.208781 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:47.209706 (+ 925us) server_negotiation.cc:299] Negotiation successful
0618 22:26:47.209874 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260618 22:26:47.211326 11588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:47.192807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35687 (local address 127.10.242.194:43609)
0618 22:26:47.193600 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:47.193639 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:47.195883 (+ 2244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:47.196670 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:47.196681 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:47.197271 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:47.197946 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:47.197964 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:47.199026 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:47.199030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:47.200241 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:47.200252 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:47.201065 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:47.201073 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:47.201315 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:47.201966 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:47.201990 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:47.204288 (+ 2298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:47.207092 (+ 2804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:47.207101 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:47.207120 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:47.207529 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:47.207976 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:47.207982 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:47.207987 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:47.208276 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:47.208794 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:47.208803 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:47.209277 (+ 474us) client_negotiation.cc:769] Sending connection context
0618 22:26:47.209618 (+ 341us) client_negotiation.cc:240] Negotiation successful
0618 22:26:47.210182 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":310,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.195@KRBTEST.COM" created.
I20260618 22:26:47.212867 11586 heartbeater.cc:347] Connected to a master server at 127.10.242.254:35687
I20260618 22:26:47.213199 11586 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:47.213901 11586 heartbeater.cc:511] Master 127.10.242.254:35687 requested a full tablet report, sending...
I20260618 22:26:47.215314 11263 ts_manager.cc:194] Registered new tserver with Master: 07d7efa6d4f24acab9947419828866dc (127.10.242.194:44695)
I20260618 22:26:47.216137 11263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:43609
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
I20260618 22:26:47.232405 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:35687
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:36315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:47.559240 11591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:47.559537 11591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:47.559651 11591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:47.567421 11591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:47.567548 11591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:47.567663 11591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:26:47.577628 11591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:36315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:35687
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:26:47.578946 11591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:47.580206 11591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:47.593577 11596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:47.593451 11597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:47.593736 11599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:47.594216 11591 server_base.cc:1034] running on GCE node
I20260618 22:26:47.594815 11591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:47.595742 11591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:47.596936 11591 hybrid_clock.cc:648] HybridClock initialized: now 1781821607596889 us; error 43 us; skew 500 ppm
I20260618 22:26:47.597285 11591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:26:47 dist-test-slave-0956 krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821607, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:47.600782 11591 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:26:47.602353 11591 webserver.cc:460] Webserver started at http://127.10.242.195:38281/ using document root <none> and password file <none>
I20260618 22:26:47.603058 11591 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:47.603168 11591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:47.603503 11591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:47.606495 11591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "cc528a227886416ab4b0ea7e557229b8"
format_stamp: "Formatted at 2026-06-18 22:26:47 on dist-test-slave-0956"
server_key: "5d8ee45b5fe750a597cf5d6ee90c8a2d"
server_key_iv: "c5d7df41d9a4771ec50113f918f059fb"
server_key_version: "encryptionkey@0"
I20260618 22:26:47.607251 11591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "cc528a227886416ab4b0ea7e557229b8"
format_stamp: "Formatted at 2026-06-18 22:26:47 on dist-test-slave-0956"
server_key: "5d8ee45b5fe750a597cf5d6ee90c8a2d"
server_key_iv: "c5d7df41d9a4771ec50113f918f059fb"
server_key_version: "encryptionkey@0"
I20260618 22:26:47.612602 11591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260618 22:26:47.616205 11606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:47.617810 11591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:26:47.617966 11591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "cc528a227886416ab4b0ea7e557229b8"
format_stamp: "Formatted at 2026-06-18 22:26:47 on dist-test-slave-0956"
server_key: "5d8ee45b5fe750a597cf5d6ee90c8a2d"
server_key_iv: "c5d7df41d9a4771ec50113f918f059fb"
server_key_version: "encryptionkey@0"
I20260618 22:26:47.618158 11591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:47.635453 11591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:47.639814 11591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:47.640134 11591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:47.641161 11591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:47.642526 11591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:47.642601 11591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:47.642716 11591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:47.642765 11591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:47.676339 11591 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:41581
I20260618 22:26:47.676380 11718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:41581 every 8 connection(s)
I20260618 22:26:47.677675 11591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:26:47.688138 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 11591
I20260618 22:26:47.688364 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:26:47.688779 11211 external_mini_cluster.cc:1428] Setting key 77a4ce7175cd7a8fbde57744c326a007
Jun 18 22:26:47 dist-test-slave-0956 krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821607, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:26:47.699769 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:47.680697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:49735 (local address 127.10.242.254:35687)
0618 22:26:47.680887 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:47.680894 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:47.682251 (+ 1357us) server_negotiation.cc:407] Connection header received
0618 22:26:47.684835 (+ 2584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:47.684840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:47.684924 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:47.685017 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:47.687174 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:47.688017 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:47.689779 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:47.690347 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:47.690449 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:47.693877 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:47.693882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:47.693885 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:47.693918 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:47.696446 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:47.697269 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:47.697273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:47.697276 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:47.697398 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:47.697825 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:47.697829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:47.697831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:47.698512 (+ 681us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:47.698672 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:47.699312 (+ 640us) server_negotiation.cc:299] Negotiation successful
0618 22:26:47.699472 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:26:47.700770 11721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:47.681206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35687 (local address 127.10.242.195:49735)
0618 22:26:47.682058 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:47.682113 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:47.684497 (+ 2384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:47.685325 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:47.685338 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:47.686132 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:47.686965 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:47.686993 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:47.688209 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:47.688225 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:47.689641 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:47.689654 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:47.690620 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:47.690630 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:47.690911 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:47.691604 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:47.691626 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:47.693681 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:47.696587 (+ 2906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:47.696595 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:47.696608 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:47.697125 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:47.697520 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:47.697525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:47.697529 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:47.697736 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:47.698660 (+ 924us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:47.698666 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:47.698996 (+ 330us) client_negotiation.cc:769] Sending connection context
0618 22:26:47.699233 (+ 237us) client_negotiation.cc:240] Negotiation successful
0618 22:26:47.699647 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":314,"threads_started":1}
I20260618 22:26:47.702235 11719 heartbeater.cc:347] Connected to a master server at 127.10.242.254:35687
I20260618 22:26:47.702525 11719 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:47.703224 11719 heartbeater.cc:511] Master 127.10.242.254:35687 requested a full tablet report, sending...
I20260618 22:26:47.704696 11263 ts_manager.cc:194] Registered new tserver with Master: cc528a227886416ab4b0ea7e557229b8 (127.10.242.195:41581)
I20260618 22:26:47.705490 11263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:49735
I20260618 22:26:47.717265 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:26:47.718029 11452 heartbeater.cc:503] Master 127.10.242.254:35687 was elected leader, sending a full tablet report...
Jun 18 22:26:47 dist-test-slave-0956 krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821607, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 18 22:26:47 dist-test-slave-0956 krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821607, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:26:47.761409 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:47.744855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41146 (local address 127.10.242.193:41639)
0618 22:26:47.745537 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:47.745547 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:47.745667 (+ 120us) server_negotiation.cc:407] Connection header received
0618 22:26:47.745873 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:47.745881 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:47.746120 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:47.746258 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:47.747667 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:47.748699 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:47.750440 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:47.751070 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:47.751184 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:47.755817 (+ 4633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:47.755832 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:47.755848 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:47.755888 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:47.758726 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:47.759385 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:47.759392 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:47.759394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:47.759510 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:47.759852 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:47.759855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:47.759857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:47.760191 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:47.760411 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:47.760812 (+ 401us) server_negotiation.cc:299] Negotiation successful
0618 22:26:47.761034 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":341,"threads_started":1}
W20260618 22:26:47.762727 11367 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41146
Jun 18 22:26:47 dist-test-slave-0956 krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 18 22:26:47 dist-test-slave-0956 krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:26:47.807184 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:47.791802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41160 (local address 127.10.242.193:41639)
0618 22:26:47.791979 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:47.791984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:47.792454 (+ 470us) server_negotiation.cc:407] Connection header received
0618 22:26:47.792592 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:47.792596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:47.792697 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:47.792788 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:47.794208 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:47.794929 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:47.796253 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:47.796746 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:47.796879 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:47.801655 (+ 4776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:47.801662 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:47.801666 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:47.801710 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:47.804479 (+ 2769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:47.805287 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:47.805293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:47.805295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:47.805452 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:47.805861 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:47.805864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:47.805866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:47.806146 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:47.806345 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:47.806746 (+ 401us) server_negotiation.cc:299] Negotiation successful
0618 22:26:47.806934 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:26:47.808580 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 11324
I20260618 22:26:47.821213 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:47.809478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41172 (local address 127.10.242.193:41639)
0618 22:26:47.809655 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:47.809659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:47.809993 (+ 334us) server_negotiation.cc:407] Connection header received
0618 22:26:47.810125 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:47.810129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:47.810188 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:47.810271 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:47.811415 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:47.812229 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:47.813568 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:47.814067 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:47.814179 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:47.816356 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:47.816361 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:47.816365 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:47.816406 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:47.818976 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:47.819625 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:47.819629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:47.819631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:47.819769 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:47.820044 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:47.820048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:47.820050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:47.820292 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:47.820463 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:47.820735 (+ 272us) server_negotiation.cc:299] Negotiation successful
0618 22:26:47.820893 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260618 22:26:47.821944 11447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:26:47.906394 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 11324
I20260618 22:26:47.919205 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 11458
Jun 18 22:26:47 dist-test-slave-0956 krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.194@KRBTEST.COM
I20260618 22:26:47.934883 11733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:47.920072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43222 (local address 127.10.242.194:44695)
0618 22:26:47.920589 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:47.920597 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:47.920621 (+ 24us) server_negotiation.cc:407] Connection header received
0618 22:26:47.920677 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:47.920684 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:47.920880 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:47.921020 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:47.922199 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:47.923146 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:47.924338 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:47.924909 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:47.925028 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:47.929111 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:47.929125 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:47.929143 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:47.929186 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:47.931984 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:47.932708 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:47.932716 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:47.932718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:47.932842 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:47.933295 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:47.933299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:47.933302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:47.933669 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:47.933969 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:47.934285 (+ 316us) server_negotiation.cc:299] Negotiation successful
0618 22:26:47.934546 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":264,"threads_started":1}
I20260618 22:26:47.936143 11581 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:26:48.018402 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 11458
I20260618 22:26:48.035761 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 11591
Jun 18 22:26:48 dist-test-slave-0956 krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.195@KRBTEST.COM
I20260618 22:26:48.052130 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:48.036773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60102 (local address 127.10.242.195:41581)
0618 22:26:48.037363 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:48.037370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:48.037399 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:26:48.037456 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:48.037463 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:48.037728 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:48.037859 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:48.039020 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:48.040097 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:48.041510 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:48.042134 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:48.042236 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:48.046329 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:48.046342 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:48.046358 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:48.046394 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:48.049323 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:48.050047 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:48.050056 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:48.050058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:48.050181 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:48.050556 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:48.050560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:48.050562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:48.050898 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:48.051220 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:48.051527 (+ 307us) server_negotiation.cc:299] Negotiation successful
0618 22:26:48.051799 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":288,"threads_started":1}
I20260618 22:26:48.053414 11714 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:26:48.133448 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 11591
I20260618 22:26:48.146130 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 11231
I20260618 22:26:48.147357 11293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:26:48.247087 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 11231
2026-06-18T22:26:48Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3834 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:48 dist-test-slave-0956 krb5kdc[11740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:48 dist-test-slave-0956 krb5kdc[11740](info): set up 2 sockets
Jun 18 22:26:48 dist-test-slave-0956 krb5kdc[11740](info): commencing operation
krb5kdc: starting...
W20260618 22:26:49.240845 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.000s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:26:49 dist-test-slave-0956 krb5kdc[11740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:26:49.405967 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:34113
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:41053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:34113
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:49.726897 11753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:49.727207 11753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:49.727337 11753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:49.734879 11753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:49.735010 11753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:49.735085 11753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:49.735164 11753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:49.735225 11753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:49.745909 11753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:41053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:34113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:34113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:26:49.747227 11753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:49.748533 11753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:49.759881 11759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:49.759943 11761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:49.760010 11758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:49.760730 11753 server_base.cc:1034] running on GCE node
I20260618 22:26:49.761531 11753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:49.762912 11753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:49.764115 11753 hybrid_clock.cc:648] HybridClock initialized: now 1781821609764065 us; error 40 us; skew 500 ppm
I20260618 22:26:49.764437 11753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:26:49 dist-test-slave-0956 krb5kdc[11740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821609, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:49.767330 11753 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:26:49.768908 11753 webserver.cc:460] Webserver started at http://127.10.242.254:33671/ using document root <none> and password file <none>
I20260618 22:26:49.769654 11753 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:49.769766 11753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:49.770076 11753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:49.773181 11753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "65d37474a54d4b04ac3c41d8ae66e36b"
format_stamp: "Formatted at 2026-06-18 22:26:49 on dist-test-slave-0956"
server_key: "857890bed89fe800eb1f75c87e2f3074"
server_key_iv: "52a7c78ae42d79b138b0a1574d2f4eb4"
server_key_version: "encryptionkey@0"
I20260618 22:26:49.773928 11753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "65d37474a54d4b04ac3c41d8ae66e36b"
format_stamp: "Formatted at 2026-06-18 22:26:49 on dist-test-slave-0956"
server_key: "857890bed89fe800eb1f75c87e2f3074"
server_key_iv: "52a7c78ae42d79b138b0a1574d2f4eb4"
server_key_version: "encryptionkey@0"
I20260618 22:26:49.779196 11753 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:26:49.782830 11768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:49.784317 11753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:26:49.784479 11753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "65d37474a54d4b04ac3c41d8ae66e36b"
format_stamp: "Formatted at 2026-06-18 22:26:49 on dist-test-slave-0956"
server_key: "857890bed89fe800eb1f75c87e2f3074"
server_key_iv: "52a7c78ae42d79b138b0a1574d2f4eb4"
server_key_version: "encryptionkey@0"
I20260618 22:26:49.784688 11753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:49.813192 11753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:49.817288 11753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:49.817556 11753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:49.834913 11753 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:34113
I20260618 22:26:49.834913 11819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:34113 every 8 connection(s)
I20260618 22:26:49.836330 11753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:49.840065 11820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:49.841691 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 11753
I20260618 22:26:49.841909 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:26:49.842689 11211 external_mini_cluster.cc:1428] Setting key af52ba94f2b5c22ac1355fe254051a5e
I20260618 22:26:49.851629 11820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b: Bootstrap starting.
I20260618 22:26:49.854839 11820 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:49.855931 11820 log.cc:824] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:26:49 dist-test-slave-0956 krb5kdc[11740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:26:49.859465 11820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b: No bootstrap required, opened a new log
I20260618 22:26:49.865062 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.844742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56270 (local address 127.10.242.254:34113)
0618 22:26:49.845671 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.845683 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.845726 (+ 43us) server_negotiation.cc:407] Connection header received
0618 22:26:49.847400 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.847423 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.847836 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.848199 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.849727 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.851009 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.852259 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.852935 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.853159 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.857761 (+ 4602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:49.857769 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:49.857785 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:49.857825 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:49.860781 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.861562 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.861569 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.861576 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.861760 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.862142 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.862146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.862148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.862720 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:49.862994 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:49.863532 (+ 538us) server_negotiation.cc:299] Negotiation successful
0618 22:26:49.863953 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":388,"threads_started":1}
I20260618 22:26:49.867197 11820 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d37474a54d4b04ac3c41d8ae66e36b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34113 } }
I20260618 22:26:49.867551 11820 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:49.867662 11820 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65d37474a54d4b04ac3c41d8ae66e36b, State: Initialized, Role: FOLLOWER
I20260618 22:26:49.868271 11820 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d37474a54d4b04ac3c41d8ae66e36b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34113 } }
I20260618 22:26:49.868475 11820 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:49.868568 11820 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:49.868691 11820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:49.870750 11820 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d37474a54d4b04ac3c41d8ae66e36b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34113 } }
I20260618 22:26:49.871234 11820 leader_election.cc:302] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65d37474a54d4b04ac3c41d8ae66e36b; no voters:
I20260618 22:26:49.871842 11820 leader_election.cc:288] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:49.871955 11825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:49.872718 11825 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b [term 1 LEADER]: Becoming Leader. State: Replica: 65d37474a54d4b04ac3c41d8ae66e36b, State: Running, Role: LEADER
I20260618 22:26:49.873224 11825 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d37474a54d4b04ac3c41d8ae66e36b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34113 } }
I20260618 22:26:49.873963 11820 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:49.875758 11826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65d37474a54d4b04ac3c41d8ae66e36b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d37474a54d4b04ac3c41d8ae66e36b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34113 } } }
I20260618 22:26:49.875768 11827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65d37474a54d4b04ac3c41d8ae66e36b. Latest consensus state: current_term: 1 leader_uuid: "65d37474a54d4b04ac3c41d8ae66e36b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d37474a54d4b04ac3c41d8ae66e36b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34113 } } }
I20260618 22:26:49.876209 11827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:49.876209 11826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:49.878361 11830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:49.883291 11830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:49.891762 11830 catalog_manager.cc:1347] Generated new cluster ID: 4ba6319b82644148b785f7a4dc139558
I20260618 22:26:49.891947 11830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:49.918779 11830 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:49.920248 11830 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:49.928994 11830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65d37474a54d4b04ac3c41d8ae66e36b: Generated new TSK 0
I20260618 22:26:49.929874 11830 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
I20260618 22:26:49.977164 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:34113
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:41053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:50.299422 11846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:50.299731 11846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:50.299839 11846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:50.307226 11846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:50.307349 11846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:50.307463 11846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:26:50.317293 11846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:41053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:34113
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:26:50.318538 11846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:50.319828 11846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:50.332347 11851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:50.332396 11852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:50.332419 11854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:50.333850 11846 server_base.cc:1034] running on GCE node
I20260618 22:26:50.334445 11846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:50.335376 11846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:50.336575 11846 hybrid_clock.cc:648] HybridClock initialized: now 1781821610336500 us; error 70 us; skew 500 ppm
I20260618 22:26:50.336877 11846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:26:50 dist-test-slave-0956 krb5kdc[11740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821610, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:50.339982 11846 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:26:50.341470 11846 webserver.cc:460] Webserver started at http://127.10.242.193:40765/ using document root <none> and password file <none>
I20260618 22:26:50.342137 11846 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:50.342252 11846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:50.342538 11846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:50.345404 11846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "677ea8d4a5ca47a0ab8dad92dda8e2c1"
format_stamp: "Formatted at 2026-06-18 22:26:50 on dist-test-slave-0956"
server_key: "9196ce3e7c320e48f002e339b3c17d20"
server_key_iv: "86c7c05940027c67ff27df71408fe046"
server_key_version: "encryptionkey@0"
I20260618 22:26:50.346145 11846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "677ea8d4a5ca47a0ab8dad92dda8e2c1"
format_stamp: "Formatted at 2026-06-18 22:26:50 on dist-test-slave-0956"
server_key: "9196ce3e7c320e48f002e339b3c17d20"
server_key_iv: "86c7c05940027c67ff27df71408fe046"
server_key_version: "encryptionkey@0"
I20260618 22:26:50.351330 11846 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:26:50.354835 11861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:50.356233 11846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:26:50.356395 11846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "677ea8d4a5ca47a0ab8dad92dda8e2c1"
format_stamp: "Formatted at 2026-06-18 22:26:50 on dist-test-slave-0956"
server_key: "9196ce3e7c320e48f002e339b3c17d20"
server_key_iv: "86c7c05940027c67ff27df71408fe046"
server_key_version: "encryptionkey@0"
I20260618 22:26:50.356577 11846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:50.385525 11846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:50.389978 11846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:50.390246 11846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:50.391199 11846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:50.392556 11846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:50.392634 11846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:50.392748 11846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:50.392808 11846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:50.422413 11846 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:37331
I20260618 22:26:50.422472 11973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:37331 every 8 connection(s)
I20260618 22:26:50.423732 11846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:26:50.425804 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 11846
I20260618 22:26:50.426015 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:26:50.426784 11211 external_mini_cluster.cc:1428] Setting key bbbce41456182462da28c91399eb570a
Jun 18 22:26:50 dist-test-slave-0956 krb5kdc[11740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821610, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.242.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.194@KRBTEST.COM" created.
I20260618 22:26:50.445592 11976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:50.426506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:42155 (local address 127.10.242.254:34113)
0618 22:26:50.427236 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:50.427242 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:50.428015 (+ 773us) server_negotiation.cc:407] Connection header received
0618 22:26:50.430110 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:50.430117 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:50.430210 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:50.430318 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:50.431983 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:50.432944 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:50.435441 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:50.435875 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:50.435985 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:50.439973 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:50.439978 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:50.439981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:50.440017 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:50.442475 (+ 2458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:50.443176 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:50.443181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:50.443183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:50.443302 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:50.443755 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:50.443760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:50.443763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:50.444017 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:50.444155 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:50.444989 (+ 834us) server_negotiation.cc:299] Negotiation successful
0618 22:26:50.445243 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":472,"threads_started":1}
I20260618 22:26:50.446601 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:50.427085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34113 (local address 127.10.242.193:42155)
0618 22:26:50.427880 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:50.427925 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:50.429815 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:50.430543 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:50.430553 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:50.431117 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:50.431793 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:50.431816 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:50.433154 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:50.433160 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:50.435255 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:50.435282 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:50.436110 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:50.436117 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:50.436452 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:50.437192 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:50.437222 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:50.439742 (+ 2520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:50.442611 (+ 2869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:50.442621 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:50.442640 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:50.443049 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:50.443423 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:50.443427 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:50.443430 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:50.443644 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:50.444144 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:50.444151 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:50.444567 (+ 416us) client_negotiation.cc:769] Sending connection context
0618 22:26:50.444859 (+ 292us) client_negotiation.cc:240] Negotiation successful
0618 22:26:50.445422 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":307,"threads_started":1}
I20260618 22:26:50.448390 11974 heartbeater.cc:347] Connected to a master server at 127.10.242.254:34113
I20260618 22:26:50.448695 11974 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:50.449467 11974 heartbeater.cc:511] Master 127.10.242.254:34113 requested a full tablet report, sending...
I20260618 22:26:50.451498 11785 ts_manager.cc:194] Registered new tserver with Master: 677ea8d4a5ca47a0ab8dad92dda8e2c1 (127.10.242.193:37331)
I20260618 22:26:50.452893 11785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:42155
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
I20260618 22:26:50.467906 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:34113
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:41053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:50.778504 11980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:50.778810 11980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:50.778932 11980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:50.786486 11980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:50.786607 11980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:50.786759 11980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:26:50.796660 11980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:41053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:34113
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:26:50.798009 11980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:50.799306 11980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:50.812536 11985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:50.812563 11988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:50.812508 11986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:50.813493 11980 server_base.cc:1034] running on GCE node
I20260618 22:26:50.814126 11980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:50.815085 11980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:50.816319 11980 hybrid_clock.cc:648] HybridClock initialized: now 1781821610816261 us; error 47 us; skew 500 ppm
I20260618 22:26:50.816709 11980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:26:50 dist-test-slave-0956 krb5kdc[11740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821610, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:50.819875 11980 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:26:50.821470 11980 webserver.cc:460] Webserver started at http://127.10.242.194:41563/ using document root <none> and password file <none>
I20260618 22:26:50.822168 11980 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:50.822283 11980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:50.822582 11980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:50.825548 11980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "53bf8f41918d49a1b6ef12d7fe8c55b8"
format_stamp: "Formatted at 2026-06-18 22:26:50 on dist-test-slave-0956"
server_key: "43dc35902c7840df488533ccdd499bfa"
server_key_iv: "5e60886daf9def31773a430650b4054a"
server_key_version: "encryptionkey@0"
I20260618 22:26:50.826320 11980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "53bf8f41918d49a1b6ef12d7fe8c55b8"
format_stamp: "Formatted at 2026-06-18 22:26:50 on dist-test-slave-0956"
server_key: "43dc35902c7840df488533ccdd499bfa"
server_key_iv: "5e60886daf9def31773a430650b4054a"
server_key_version: "encryptionkey@0"
I20260618 22:26:50.831624 11980 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:26:50.835379 11995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:50.837003 11980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:26:50.837196 11980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "53bf8f41918d49a1b6ef12d7fe8c55b8"
format_stamp: "Formatted at 2026-06-18 22:26:50 on dist-test-slave-0956"
server_key: "43dc35902c7840df488533ccdd499bfa"
server_key_iv: "5e60886daf9def31773a430650b4054a"
server_key_version: "encryptionkey@0"
I20260618 22:26:50.837425 11980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:50.860925 11980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:50.865523 11980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:50.865775 11980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:50.866663 11980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:50.867981 11980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:50.868050 11980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:50.868141 11980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:50.868182 11980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:50.899644 11980 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:34865
I20260618 22:26:50.899708 12107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:34865 every 8 connection(s)
I20260618 22:26:50.900918 11980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:26:50.904980 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 11980
I20260618 22:26:50.905206 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:26:50.905767 11211 external_mini_cluster.cc:1428] Setting key 69f61fba06526af562af19e6f763b1d0
Jun 18 22:26:50 dist-test-slave-0956 krb5kdc[11740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821610, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.242.195@KRBTEST.COM; defaulting to no policy
I20260618 22:26:50.923156 11976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:50.903513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:33727 (local address 127.10.242.254:34113)
0618 22:26:50.903685 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:50.903690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:50.905014 (+ 1324us) server_negotiation.cc:407] Connection header received
0618 22:26:50.907340 (+ 2326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:50.907344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:50.907409 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:50.907487 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:50.909473 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:50.910173 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:50.912250 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:50.912647 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:50.912779 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:50.916846 (+ 4067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:50.916851 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:50.916855 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:50.916890 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:50.919343 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:50.920422 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:50.920427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:50.920429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:50.920620 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:50.921193 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:50.921197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:50.921199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:50.921517 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:50.921696 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:50.922668 (+ 972us) server_negotiation.cc:299] Negotiation successful
0618 22:26:50.922863 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:26:50.924203 12110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:50.904002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34113 (local address 127.10.242.194:33727)
0618 22:26:50.904788 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:50.904844 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:50.907031 (+ 2187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:50.907729 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:50.907738 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:50.908470 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:50.909301 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:50.909327 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:50.910322 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:50.910327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:50.912090 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:50.912104 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:50.912904 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:50.912911 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:50.913237 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:50.913979 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:50.914008 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:50.916581 (+ 2573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:50.919512 (+ 2931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:50.919523 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:50.919544 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:50.920253 (+ 709us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:50.920740 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:50.920745 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:50.920749 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:50.921004 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:50.921709 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:50.921718 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:50.922201 (+ 483us) client_negotiation.cc:769] Sending connection context
0618 22:26:50.922517 (+ 316us) client_negotiation.cc:240] Negotiation successful
0618 22:26:50.922956 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":266,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.195@KRBTEST.COM" created.
I20260618 22:26:50.925679 12108 heartbeater.cc:347] Connected to a master server at 127.10.242.254:34113
I20260618 22:26:50.926023 12108 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:50.926972 12108 heartbeater.cc:511] Master 127.10.242.254:34113 requested a full tablet report, sending...
I20260618 22:26:50.928632 11785 ts_manager.cc:194] Registered new tserver with Master: 53bf8f41918d49a1b6ef12d7fe8c55b8 (127.10.242.194:34865)
I20260618 22:26:50.929627 11785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:33727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
I20260618 22:26:50.945991 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:34113
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:41053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:51.259740 12113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:51.260138 12113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:51.260269 12113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:51.267436 12113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:51.267547 12113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:51.267696 12113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:26:51.277371 12113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:41053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:34113
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:26:51.278633 12113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:51.279963 12113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:51.292744 12118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:51.292851 12121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:51.292906 12119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:51.294047 12113 server_base.cc:1034] running on GCE node
I20260618 22:26:51.294646 12113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:51.295559 12113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:51.296747 12113 hybrid_clock.cc:648] HybridClock initialized: now 1781821611296695 us; error 47 us; skew 500 ppm
I20260618 22:26:51.297045 12113 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:26:51 dist-test-slave-0956 krb5kdc[11740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821611, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:51.305454 12113 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:26:51.307135 12113 webserver.cc:460] Webserver started at http://127.10.242.195:37133/ using document root <none> and password file <none>
I20260618 22:26:51.307837 12113 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:51.307955 12113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:51.308257 12113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:51.311444 12113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "9c8edec360344964a70e4943fda51beb"
format_stamp: "Formatted at 2026-06-18 22:26:51 on dist-test-slave-0956"
server_key: "08a0f8f783dbd87139f7f9c2772c5ce8"
server_key_iv: "1bb0168df7ee38049d329f3228e89236"
server_key_version: "encryptionkey@0"
I20260618 22:26:51.312201 12113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "9c8edec360344964a70e4943fda51beb"
format_stamp: "Formatted at 2026-06-18 22:26:51 on dist-test-slave-0956"
server_key: "08a0f8f783dbd87139f7f9c2772c5ce8"
server_key_iv: "1bb0168df7ee38049d329f3228e89236"
server_key_version: "encryptionkey@0"
I20260618 22:26:51.317301 12113 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260618 22:26:51.320894 12128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:51.322309 12113 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:26:51.322466 12113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "9c8edec360344964a70e4943fda51beb"
format_stamp: "Formatted at 2026-06-18 22:26:51 on dist-test-slave-0956"
server_key: "08a0f8f783dbd87139f7f9c2772c5ce8"
server_key_iv: "1bb0168df7ee38049d329f3228e89236"
server_key_version: "encryptionkey@0"
I20260618 22:26:51.322656 12113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:51.341898 12113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:51.346096 12113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:51.346378 12113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:51.347381 12113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:51.348785 12113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:51.348869 12113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:51.348999 12113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:51.349056 12113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:51.379927 12113 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:42183
I20260618 22:26:51.379972 12240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:42183 every 8 connection(s)
I20260618 22:26:51.381320 12113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:26:51.382350 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 12113
I20260618 22:26:51.382510 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:26:51.382958 11211 external_mini_cluster.cc:1428] Setting key 228ad2dda9f1f25b13ddd3e85d0676c2
Jun 18 22:26:51 dist-test-slave-0956 krb5kdc[11740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821611, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:26:51.402211 11976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.383993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:40555 (local address 127.10.242.254:34113)
0618 22:26:51.384173 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:51.384178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:51.386232 (+ 2054us) server_negotiation.cc:407] Connection header received
0618 22:26:51.388026 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:51.388031 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:51.388144 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:51.388224 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:51.389878 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.390617 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.392153 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.392550 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.392666 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:51.396439 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:51.396445 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:51.396450 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:51.396492 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:51.399210 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:51.399913 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:51.399917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:51.399919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:51.400032 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:51.400431 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:51.400435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:51.400437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:51.400706 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:51.400843 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:51.401630 (+ 787us) server_negotiation.cc:299] Negotiation successful
0618 22:26:51.401865 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260618 22:26:51.403260 12243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.384665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34113 (local address 127.10.242.195:40555)
0618 22:26:51.386092 (+ 1427us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:51.386131 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:51.387776 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:51.388494 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:51.388503 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:51.389056 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:51.389719 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:51.389739 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.390752 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.390756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:51.392011 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:51.392022 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.392782 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.392789 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:51.393008 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:51.394148 (+ 1140us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:51.394174 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:51.396222 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:51.399348 (+ 3126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:51.399358 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:51.399376 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:51.399794 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:51.400132 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:51.400137 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:51.400141 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:51.400323 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:51.400823 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:51.400831 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:51.401256 (+ 425us) client_negotiation.cc:769] Sending connection context
0618 22:26:51.401512 (+ 256us) client_negotiation.cc:240] Negotiation successful
0618 22:26:51.402106 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1170,"thread_start_us":956,"threads_started":1}
I20260618 22:26:51.404707 12241 heartbeater.cc:347] Connected to a master server at 127.10.242.254:34113
I20260618 22:26:51.404973 12241 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:51.405701 12241 heartbeater.cc:511] Master 127.10.242.254:34113 requested a full tablet report, sending...
I20260618 22:26:51.407122 11785 ts_manager.cc:194] Registered new tserver with Master: 9c8edec360344964a70e4943fda51beb (127.10.242.195:42183)
I20260618 22:26:51.407927 11785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:40555
I20260618 22:26:51.410705 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:26:51.429777 11976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.416535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56272 (local address 127.10.242.254:34113)
0618 22:26:51.416680 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:51.416685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:51.417150 (+ 465us) server_negotiation.cc:407] Connection header received
0618 22:26:51.417327 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:51.417330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:51.417391 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:51.417470 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:51.419233 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.419942 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.421172 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.421617 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.421738 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:51.423995 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:51.424002 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:51.424006 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:51.424046 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:51.427359 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:51.428025 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:51.428029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:51.428031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:51.428164 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:51.428514 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:51.428519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:51.428521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:51.428864 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:51.429040 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:51.429377 (+ 337us) server_negotiation.cc:299] Negotiation successful
0618 22:26:51.429547 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:26:51.436095 11785 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56272:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:26:51.443442 11785 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:26:51.456287 11974 heartbeater.cc:503] Master 127.10.242.254:34113 was elected leader, sending a full tablet report...
I20260618 22:26:51.480358 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.456542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60120 (local address 127.10.242.194:34865)
0618 22:26:51.469491 (+ 12949us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:51.469500 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:51.470693 (+ 1193us) server_negotiation.cc:407] Connection header received
0618 22:26:51.470811 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:51.470819 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:51.471071 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:51.471280 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:51.473396 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.475042 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.477650 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.478993 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.479129 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:51.479254 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:51.479853 (+ 599us) server_negotiation.cc:299] Negotiation successful
0618 22:26:51.480020 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12811,"thread_start_us":12662,"threads_started":1}
I20260618 22:26:51.480634 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.456632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36042 (local address 127.10.242.195:42183)
0618 22:26:51.469105 (+ 12473us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:51.469115 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:51.470312 (+ 1197us) server_negotiation.cc:407] Connection header received
0618 22:26:51.470435 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:51.470443 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:51.470729 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:51.470937 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:51.473158 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.474916 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.477189 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.478637 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.478837 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:51.479049 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:51.480010 (+ 961us) server_negotiation.cc:299] Negotiation successful
0618 22:26:51.480217 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12262,"thread_start_us":12107,"threads_started":1}
I20260618 22:26:51.481032 12254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.455976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60672 (local address 127.10.242.193:37331)
0618 22:26:51.470945 (+ 14969us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:51.470955 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:51.470983 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:26:51.471073 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:51.471081 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:51.471299 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:51.471774 (+ 475us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:51.473338 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.474700 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.477416 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.478821 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.479009 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:51.479186 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:51.480540 (+ 1354us) server_negotiation.cc:299] Negotiation successful
0618 22:26:51.480714 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14777,"thread_start_us":15269,"threads_started":1}
I20260618 22:26:51.481207 12253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.455812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:37331 (local address 127.0.0.1:60672)
0618 22:26:51.467342 (+ 11530us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:51.467408 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:51.467643 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:51.471597 (+ 3954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:51.471605 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:51.471641 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:51.473115 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:51.473128 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.474817 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.474822 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:51.477228 (+ 2406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:51.477242 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.479212 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.479220 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:51.479404 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:51.479439 (+ 35us) client_negotiation.cc:769] Sending connection context
0618 22:26:51.479515 (+ 76us) client_negotiation.cc:240] Negotiation successful
0618 22:26:51.479641 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11193,"mutex_wait_us":149,"thread_start_us":12044,"threads_started":1}
I20260618 22:26:51.481462 12255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.456061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:42183 (local address 127.0.0.1:36042)
0618 22:26:51.469947 (+ 13886us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:51.469970 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:51.470153 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:51.471255 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:51.471263 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:51.471295 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:51.472837 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:51.472855 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.475103 (+ 2248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.475148 (+ 45us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:51.476932 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:51.476949 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.478830 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.478837 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:51.479045 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:51.479102 (+ 57us) client_negotiation.cc:769] Sending connection context
0618 22:26:51.479252 (+ 150us) client_negotiation.cc:240] Negotiation successful
0618 22:26:51.479469 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13744,"mutex_wait_us":2,"thread_start_us":13600,"threads_started":1}
I20260618 22:26:51.481863 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.456343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:34865 (local address 127.0.0.1:60120)
0618 22:26:51.470366 (+ 14023us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:51.470386 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:51.470537 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:51.471842 (+ 1305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:51.471846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:51.471876 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:51.473198 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:51.473212 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.475164 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.475169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:51.477469 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:51.477481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.479438 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.479452 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:51.479651 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:51.479685 (+ 34us) client_negotiation.cc:769] Sending connection context
0618 22:26:51.479772 (+ 87us) client_negotiation.cc:240] Negotiation successful
0618 22:26:51.479898 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13886,"mutex_wait_us":64,"thread_start_us":13717,"threads_started":1}
I20260618 22:26:51.484905 11909 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa22bafd33344c68ae173c609f48b08 (DEFAULT_TABLE table=test-table [id=815a22735f044e75b43e9a6d30855cbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:51.485314 12043 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa22bafd33344c68ae173c609f48b08 (DEFAULT_TABLE table=test-table [id=815a22735f044e75b43e9a6d30855cbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:51.485314 12176 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa22bafd33344c68ae173c609f48b08 (DEFAULT_TABLE table=test-table [id=815a22735f044e75b43e9a6d30855cbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:51.486245 11909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa22bafd33344c68ae173c609f48b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:51.486613 12176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa22bafd33344c68ae173c609f48b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:51.486624 12043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa22bafd33344c68ae173c609f48b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:51.498652 12259 tablet_bootstrap.cc:492] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1: Bootstrap starting.
I20260618 22:26:51.501578 12261 tablet_bootstrap.cc:492] T 4aa22bafd33344c68ae173c609f48b08 P 9c8edec360344964a70e4943fda51beb: Bootstrap starting.
I20260618 22:26:51.503278 12259 tablet_bootstrap.cc:654] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:51.504599 12259 log.cc:824] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:51.505030 12261 tablet_bootstrap.cc:654] T 4aa22bafd33344c68ae173c609f48b08 P 9c8edec360344964a70e4943fda51beb: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:51.506110 12260 tablet_bootstrap.cc:492] T 4aa22bafd33344c68ae173c609f48b08 P 53bf8f41918d49a1b6ef12d7fe8c55b8: Bootstrap starting.
I20260618 22:26:51.506812 12261 log.cc:824] T 4aa22bafd33344c68ae173c609f48b08 P 9c8edec360344964a70e4943fda51beb: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:51.507735 12259 tablet_bootstrap.cc:492] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1: No bootstrap required, opened a new log
I20260618 22:26:51.508015 12259 ts_tablet_manager.cc:1397] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260618 22:26:51.509954 12260 tablet_bootstrap.cc:654] T 4aa22bafd33344c68ae173c609f48b08 P 53bf8f41918d49a1b6ef12d7fe8c55b8: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:51.510063 12261 tablet_bootstrap.cc:492] T 4aa22bafd33344c68ae173c609f48b08 P 9c8edec360344964a70e4943fda51beb: No bootstrap required, opened a new log
I20260618 22:26:51.510363 12261 ts_tablet_manager.cc:1397] T 4aa22bafd33344c68ae173c609f48b08 P 9c8edec360344964a70e4943fda51beb: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260618 22:26:51.511137 12260 log.cc:824] T 4aa22bafd33344c68ae173c609f48b08 P 53bf8f41918d49a1b6ef12d7fe8c55b8: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:51.514248 12260 tablet_bootstrap.cc:492] T 4aa22bafd33344c68ae173c609f48b08 P 53bf8f41918d49a1b6ef12d7fe8c55b8: No bootstrap required, opened a new log
I20260618 22:26:51.514513 12260 ts_tablet_manager.cc:1397] T 4aa22bafd33344c68ae173c609f48b08 P 53bf8f41918d49a1b6ef12d7fe8c55b8: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260618 22:26:51.516259 12259 raft_consensus.cc:348] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "677ea8d4a5ca47a0ab8dad92dda8e2c1" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 37331 } } peers { permanent_uuid: "9c8edec360344964a70e4943fda51beb" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42183 } } peers { permanent_uuid: "53bf8f41918d49a1b6ef12d7fe8c55b8" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34865 } }
I20260618 22:26:51.516649 12259 raft_consensus.cc:374] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:51.516767 12259 raft_consensus.cc:729] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 677ea8d4a5ca47a0ab8dad92dda8e2c1, State: Initialized, Role: FOLLOWER
I20260618 22:26:51.517349 12259 consensus_queue.cc:260] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "677ea8d4a5ca47a0ab8dad92dda8e2c1" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 37331 } } peers { permanent_uuid: "9c8edec360344964a70e4943fda51beb" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42183 } } peers { permanent_uuid: "53bf8f41918d49a1b6ef12d7fe8c55b8" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34865 } }
I20260618 22:26:51.518981 12259 ts_tablet_manager.cc:1428] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260618 22:26:51.520391 12261 raft_consensus.cc:348] T 4aa22bafd33344c68ae173c609f48b08 P 9c8edec360344964a70e4943fda51beb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "677ea8d4a5ca47a0ab8dad92dda8e2c1" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 37331 } } peers { permanent_uuid: "9c8edec360344964a70e4943fda51beb" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42183 } } peers { permanent_uuid: "53bf8f41918d49a1b6ef12d7fe8c55b8" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34865 } }
I20260618 22:26:51.520870 12261 raft_consensus.cc:374] T 4aa22bafd33344c68ae173c609f48b08 P 9c8edec360344964a70e4943fda51beb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:51.521006 12261 raft_consensus.cc:729] T 4aa22bafd33344c68ae173c609f48b08 P 9c8edec360344964a70e4943fda51beb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8edec360344964a70e4943fda51beb, State: Initialized, Role: FOLLOWER
I20260618 22:26:51.521683 12261 consensus_queue.cc:260] T 4aa22bafd33344c68ae173c609f48b08 P 9c8edec360344964a70e4943fda51beb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "677ea8d4a5ca47a0ab8dad92dda8e2c1" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 37331 } } peers { permanent_uuid: "9c8edec360344964a70e4943fda51beb" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42183 } } peers { permanent_uuid: "53bf8f41918d49a1b6ef12d7fe8c55b8" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34865 } }
I20260618 22:26:51.525774 12241 heartbeater.cc:503] Master 127.10.242.254:34113 was elected leader, sending a full tablet report...
I20260618 22:26:51.526485 12261 ts_tablet_manager.cc:1428] T 4aa22bafd33344c68ae173c609f48b08 P 9c8edec360344964a70e4943fda51beb: Time spent starting tablet: real 0.016s user 0.008s sys 0.007s
I20260618 22:26:51.525238 12260 raft_consensus.cc:348] T 4aa22bafd33344c68ae173c609f48b08 P 53bf8f41918d49a1b6ef12d7fe8c55b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "677ea8d4a5ca47a0ab8dad92dda8e2c1" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 37331 } } peers { permanent_uuid: "9c8edec360344964a70e4943fda51beb" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42183 } } peers { permanent_uuid: "53bf8f41918d49a1b6ef12d7fe8c55b8" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34865 } }
I20260618 22:26:51.527038 12260 raft_consensus.cc:374] T 4aa22bafd33344c68ae173c609f48b08 P 53bf8f41918d49a1b6ef12d7fe8c55b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:51.527153 12260 raft_consensus.cc:729] T 4aa22bafd33344c68ae173c609f48b08 P 53bf8f41918d49a1b6ef12d7fe8c55b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53bf8f41918d49a1b6ef12d7fe8c55b8, State: Initialized, Role: FOLLOWER
I20260618 22:26:51.527666 12260 consensus_queue.cc:260] T 4aa22bafd33344c68ae173c609f48b08 P 53bf8f41918d49a1b6ef12d7fe8c55b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "677ea8d4a5ca47a0ab8dad92dda8e2c1" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 37331 } } peers { permanent_uuid: "9c8edec360344964a70e4943fda51beb" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42183 } } peers { permanent_uuid: "53bf8f41918d49a1b6ef12d7fe8c55b8" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34865 } }
I20260618 22:26:51.528661 12108 heartbeater.cc:503] Master 127.10.242.254:34113 was elected leader, sending a full tablet report...
I20260618 22:26:51.529333 12260 ts_tablet_manager.cc:1428] T 4aa22bafd33344c68ae173c609f48b08 P 53bf8f41918d49a1b6ef12d7fe8c55b8: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
W20260618 22:26:51.633288 12242 tablet.cc:2369] T 4aa22bafd33344c68ae173c609f48b08 P 9c8edec360344964a70e4943fda51beb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:26:51.652872 12109 tablet.cc:2369] T 4aa22bafd33344c68ae173c609f48b08 P 53bf8f41918d49a1b6ef12d7fe8c55b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:26:51.675935 11975 tablet.cc:2369] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:26:51.709146 12265 raft_consensus.cc:482] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:26:51.709432 12265 raft_consensus.cc:504] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "677ea8d4a5ca47a0ab8dad92dda8e2c1" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 37331 } } peers { permanent_uuid: "9c8edec360344964a70e4943fda51beb" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42183 } } peers { permanent_uuid: "53bf8f41918d49a1b6ef12d7fe8c55b8" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34865 } }
I20260618 22:26:51.711058 12265 leader_election.cc:288] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8edec360344964a70e4943fda51beb (127.10.242.195:42183), 53bf8f41918d49a1b6ef12d7fe8c55b8 (127.10.242.194:34865)
I20260618 22:26:51.720229 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.711509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:38385 (local address 127.10.242.194:34865)
0618 22:26:51.711707 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:51.711714 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:51.712952 (+ 1238us) server_negotiation.cc:407] Connection header received
0618 22:26:51.713190 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:51.713196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:51.713273 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:51.713378 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:51.714127 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.715203 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.718014 (+ 2811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.719186 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.719346 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:51.719417 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:51.719847 (+ 430us) server_negotiation.cc:299] Negotiation successful
0618 22:26:51.719974 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260618 22:26:51.720229 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.711506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:38693 (local address 127.10.242.195:42183)
0618 22:26:51.711707 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:51.711714 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:51.712993 (+ 1279us) server_negotiation.cc:407] Connection header received
0618 22:26:51.713190 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:51.713196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:51.713298 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:51.713377 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:51.714133 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.715199 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.718028 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.719189 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.719361 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:51.719420 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:51.719841 (+ 421us) server_negotiation.cc:299] Negotiation successful
0618 22:26:51.719974 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:26:51.720487 12269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.711600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:42183 (local address 127.10.242.193:38693)
0618 22:26:51.712898 (+ 1298us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:51.712912 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:51.713019 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:51.713400 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:51.713406 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:51.713460 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:51.713947 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:51.713964 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.715342 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.715347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:51.717863 (+ 2516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:51.717879 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.719426 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.719432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:51.719639 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:51.719700 (+ 61us) client_negotiation.cc:769] Sending connection context
0618 22:26:51.719795 (+ 95us) client_negotiation.cc:240] Negotiation successful
0618 22:26:51.719985 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1181,"mutex_wait_us":206,"spinlock_wait_cycles":67456,"thread_start_us":864,"threads_started":1}
I20260618 22:26:51.720640 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.711600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:34865 (local address 127.10.242.193:38385)
0618 22:26:51.712827 (+ 1227us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:51.712847 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:51.713019 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:51.713400 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:51.713406 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:51.713468 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:51.713991 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:51.714001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.715344 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.715348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:51.717863 (+ 2515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:51.717879 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.719442 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.719447 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:51.719675 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:51.719721 (+ 46us) client_negotiation.cc:769] Sending connection context
0618 22:26:51.719795 (+ 74us) client_negotiation.cc:240] Negotiation successful
0618 22:26:51.719985 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1082,"mutex_wait_us":213,"spinlock_wait_cycles":31616,"thread_start_us":858,"threads_started":1}
I20260618 22:26:51.721594 12196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa22bafd33344c68ae173c609f48b08" candidate_uuid: "677ea8d4a5ca47a0ab8dad92dda8e2c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8edec360344964a70e4943fda51beb" is_pre_election: true
I20260618 22:26:51.721593 12063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa22bafd33344c68ae173c609f48b08" candidate_uuid: "677ea8d4a5ca47a0ab8dad92dda8e2c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53bf8f41918d49a1b6ef12d7fe8c55b8" is_pre_election: true
I20260618 22:26:51.722146 12063 raft_consensus.cc:2454] T 4aa22bafd33344c68ae173c609f48b08 P 53bf8f41918d49a1b6ef12d7fe8c55b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 677ea8d4a5ca47a0ab8dad92dda8e2c1 in term 0.
I20260618 22:26:51.722146 12196 raft_consensus.cc:2454] T 4aa22bafd33344c68ae173c609f48b08 P 9c8edec360344964a70e4943fda51beb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 677ea8d4a5ca47a0ab8dad92dda8e2c1 in term 0.
I20260618 22:26:51.722832 11865 leader_election.cc:302] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 677ea8d4a5ca47a0ab8dad92dda8e2c1, 9c8edec360344964a70e4943fda51beb; no voters:
I20260618 22:26:51.723208 12265 raft_consensus.cc:2781] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:26:51.723361 12265 raft_consensus.cc:482] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:26:51.723462 12265 raft_consensus.cc:3037] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:51.726082 12265 raft_consensus.cc:504] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "677ea8d4a5ca47a0ab8dad92dda8e2c1" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 37331 } } peers { permanent_uuid: "9c8edec360344964a70e4943fda51beb" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42183 } } peers { permanent_uuid: "53bf8f41918d49a1b6ef12d7fe8c55b8" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34865 } }
I20260618 22:26:51.726703 12265 leader_election.cc:288] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1 [CANDIDATE]: Term 1 election: Requested vote from peers 9c8edec360344964a70e4943fda51beb (127.10.242.195:42183), 53bf8f41918d49a1b6ef12d7fe8c55b8 (127.10.242.194:34865)
I20260618 22:26:51.727278 12196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa22bafd33344c68ae173c609f48b08" candidate_uuid: "677ea8d4a5ca47a0ab8dad92dda8e2c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8edec360344964a70e4943fda51beb"
I20260618 22:26:51.727344 12063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa22bafd33344c68ae173c609f48b08" candidate_uuid: "677ea8d4a5ca47a0ab8dad92dda8e2c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53bf8f41918d49a1b6ef12d7fe8c55b8"
I20260618 22:26:51.727481 12196 raft_consensus.cc:3037] T 4aa22bafd33344c68ae173c609f48b08 P 9c8edec360344964a70e4943fda51beb [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:51.727531 12063 raft_consensus.cc:3037] T 4aa22bafd33344c68ae173c609f48b08 P 53bf8f41918d49a1b6ef12d7fe8c55b8 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:51.729763 12196 raft_consensus.cc:2454] T 4aa22bafd33344c68ae173c609f48b08 P 9c8edec360344964a70e4943fda51beb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 677ea8d4a5ca47a0ab8dad92dda8e2c1 in term 1.
I20260618 22:26:51.729837 12063 raft_consensus.cc:2454] T 4aa22bafd33344c68ae173c609f48b08 P 53bf8f41918d49a1b6ef12d7fe8c55b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 677ea8d4a5ca47a0ab8dad92dda8e2c1 in term 1.
I20260618 22:26:51.730161 11865 leader_election.cc:302] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 677ea8d4a5ca47a0ab8dad92dda8e2c1, 9c8edec360344964a70e4943fda51beb; no voters:
I20260618 22:26:51.730468 12265 raft_consensus.cc:2781] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:51.730878 12265 raft_consensus.cc:686] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1 [term 1 LEADER]: Becoming Leader. State: Replica: 677ea8d4a5ca47a0ab8dad92dda8e2c1, State: Running, Role: LEADER
I20260618 22:26:51.731338 12265 consensus_queue.cc:237] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "677ea8d4a5ca47a0ab8dad92dda8e2c1" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 37331 } } peers { permanent_uuid: "9c8edec360344964a70e4943fda51beb" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42183 } } peers { permanent_uuid: "53bf8f41918d49a1b6ef12d7fe8c55b8" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34865 } }
I20260618 22:26:51.736071 11784 catalog_manager.cc:5495] T 4aa22bafd33344c68ae173c609f48b08 P 677ea8d4a5ca47a0ab8dad92dda8e2c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 677ea8d4a5ca47a0ab8dad92dda8e2c1 (127.10.242.193). New cstate: current_term: 1 leader_uuid: "677ea8d4a5ca47a0ab8dad92dda8e2c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "677ea8d4a5ca47a0ab8dad92dda8e2c1" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 37331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8edec360344964a70e4943fda51beb" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53bf8f41918d49a1b6ef12d7fe8c55b8" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34865 } health_report { overall_health: UNKNOWN } } }
Jun 18 22:26:51 dist-test-slave-0956 krb5kdc[11740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:26:51.780627 12254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.766699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60676 (local address 127.10.242.193:37331)
0618 22:26:51.766920 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:51.766925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:51.767288 (+ 363us) server_negotiation.cc:407] Connection header received
0618 22:26:51.767449 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:51.767453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:51.767524 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:51.767604 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:51.769037 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.769821 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.771047 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.771472 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.771579 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:51.775383 (+ 3804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:51.775399 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:51.775415 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:51.775451 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:51.778068 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:51.778680 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:51.778688 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:51.778690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:51.778840 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:51.779171 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:51.779174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:51.779176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:51.779531 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:51.779800 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:51.780092 (+ 292us) server_negotiation.cc:299] Negotiation successful
0618 22:26:51.780337 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
Jun 18 22:26:51 dist-test-slave-0956 krb5kdc[11740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821611, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 18 22:26:51 dist-test-slave-0956 krb5kdc[11740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821611, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:26:51.823426 12254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.809820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60690 (local address 127.10.242.193:37331)
0618 22:26:51.809989 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:51.809996 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:51.810369 (+ 373us) server_negotiation.cc:407] Connection header received
0618 22:26:51.810543 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:51.810548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:51.810612 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:51.810693 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:51.812044 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.812742 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.814156 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.814579 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.814691 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:51.818414 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:51.818419 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:51.818423 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:51.818461 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:51.820895 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:51.821562 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:51.821566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:51.821569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:51.821696 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:51.822118 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:51.822122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:51.822124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:51.822378 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:51.822631 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:51.822893 (+ 262us) server_negotiation.cc:299] Negotiation successful
0618 22:26:51.823105 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Jun 18 22:26:51 dist-test-slave-0956 krb5kdc[11740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 18 22:26:51 dist-test-slave-0956 krb5kdc[11740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:26:51.868487 12254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.854659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60694 (local address 127.10.242.193:37331)
0618 22:26:51.854876 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:51.854882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:51.855128 (+ 246us) server_negotiation.cc:407] Connection header received
0618 22:26:51.855301 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:51.855305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:51.855385 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:51.855471 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:51.856993 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.857855 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.859062 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.859492 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.859607 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:51.863576 (+ 3969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:51.863582 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:51.863585 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:51.863623 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:51.866051 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:51.866703 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:51.866708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:51.866710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:51.866839 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:51.867174 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:51.867177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:51.867179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:51.867430 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:51.867668 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:51.867920 (+ 252us) server_negotiation.cc:299] Negotiation successful
0618 22:26:51.868147 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260618 22:26:51.870270 11889 tablet_service.cc:2908] Rejecting scan request for tablet 4aa22bafd33344c68ae173c609f48b08: Uninitialized: clean time has not yet been initialized
I20260618 22:26:51.872886 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 11846
I20260618 22:26:51.886386 12254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.873919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60710 (local address 127.10.242.193:37331)
0618 22:26:51.874125 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:51.874131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:51.874344 (+ 213us) server_negotiation.cc:407] Connection header received
0618 22:26:51.874494 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:51.874498 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:51.874552 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:51.874641 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:51.875885 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.876623 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.877840 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.878245 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.878346 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:51.881236 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:51.881241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:51.881245 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:51.881281 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:51.883864 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:51.884562 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:51.884567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:51.884569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:51.884745 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:51.885053 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:51.885056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:51.885058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:51.885350 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:51.885530 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:51.885877 (+ 347us) server_negotiation.cc:299] Negotiation successful
0618 22:26:51.886068 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:26:51.886965 11969 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:26:51.978092 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 11846
I20260618 22:26:51.991273 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 11980
Jun 18 22:26:51 dist-test-slave-0956 krb5kdc[11740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.194@KRBTEST.COM
I20260618 22:26:52.005586 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.992121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60130 (local address 127.10.242.194:34865)
0618 22:26:51.992281 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:51.992285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:51.992304 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:26:51.992383 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:51.992386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:51.992447 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:51.992525 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:51.993726 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.994623 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.995908 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:51.996336 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:51.996438 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.000339 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:52.000355 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:52.000371 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:52.000412 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:52.003113 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.003791 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.003798 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.003801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.003926 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.004233 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.004236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.004238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.004609 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:52.004847 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:52.005021 (+ 174us) server_negotiation.cc:299] Negotiation successful
0618 22:26:52.005274 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:26:52.006258 12103 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:26:52.093954 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 11980
I20260618 22:26:52.106503 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 12113
Jun 18 22:26:52 dist-test-slave-0956 krb5kdc[11740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.195@KRBTEST.COM
I20260618 22:26:52.121603 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.107232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36058 (local address 127.10.242.195:42183)
0618 22:26:52.107409 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.107414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.107435 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:26:52.107588 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.107594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.107662 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.107782 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.109186 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.109974 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.111352 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.111817 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.111917 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.116177 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:52.116193 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:52.116207 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:52.116251 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:52.118783 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.119431 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.119439 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.119441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.119578 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.119971 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.119976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.119979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.120395 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:52.120639 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:52.121032 (+ 393us) server_negotiation.cc:299] Negotiation successful
0618 22:26:52.121289 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:26:52.122213 12236 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:26:52.207659 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 12113
I20260618 22:26:52.219992 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 11753
I20260618 22:26:52.221231 11815 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:26:52.326035 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 11753
2026-06-18T22:26:52Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4079 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:52 dist-test-slave-0956 krb5kdc[12289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:52 dist-test-slave-0956 krb5kdc[12289](info): set up 2 sockets
Jun 18 22:26:52 dist-test-slave-0956 krb5kdc[12289](info): commencing operation
krb5kdc: starting...
W20260618 22:26:53.316787 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.953s user 0.000s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:26:53 dist-test-slave-0956 krb5kdc[12289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:26:53.488745 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:46085
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:41123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:46085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:53.802991 12302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:53.803299 12302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:53.803436 12302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:53.810696 12302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:53.810814 12302 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260618 22:26:53.810858 12302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:53.810937 12302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:53.810999 12302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:53.811066 12302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:53.820909 12302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:41123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:46085
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:46085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:26:53.822225 12302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:53.823510 12302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:53.834967 12307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:53.834918 12310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:53.834946 12308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:53.835826 12302 server_base.cc:1034] running on GCE node
I20260618 22:26:53.836580 12302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:53.838019 12302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:53.839227 12302 hybrid_clock.cc:648] HybridClock initialized: now 1781821613839153 us; error 55 us; skew 500 ppm
I20260618 22:26:53.839541 12302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:26:53 dist-test-slave-0956 krb5kdc[12289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821613, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:53.842716 12302 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:26:53.844197 12302 webserver.cc:460] Webserver started at http://127.10.242.254:39053/ using document root <none> and password file <none>
I20260618 22:26:53.844897 12302 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:53.845007 12302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:53.845350 12302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:53.848263 12302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "69805a4c96284ab6af68e887cdd744f7"
format_stamp: "Formatted at 2026-06-18 22:26:53 on dist-test-slave-0956"
server_key: "071a5801cf908abdd5112bde653130cd"
server_key_iv: "2046e718c07d4557f58e51827ebe5cb1"
server_key_version: "encryptionkey@0"
I20260618 22:26:53.849023 12302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "69805a4c96284ab6af68e887cdd744f7"
format_stamp: "Formatted at 2026-06-18 22:26:53 on dist-test-slave-0956"
server_key: "071a5801cf908abdd5112bde653130cd"
server_key_iv: "2046e718c07d4557f58e51827ebe5cb1"
server_key_version: "encryptionkey@0"
I20260618 22:26:53.854250 12302 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:26:53.857839 12317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:53.859306 12302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:26:53.859483 12302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "69805a4c96284ab6af68e887cdd744f7"
format_stamp: "Formatted at 2026-06-18 22:26:53 on dist-test-slave-0956"
server_key: "071a5801cf908abdd5112bde653130cd"
server_key_iv: "2046e718c07d4557f58e51827ebe5cb1"
server_key_version: "encryptionkey@0"
I20260618 22:26:53.859717 12302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:53.884454 12302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:53.888774 12302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:53.889017 12302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:53.910221 12302 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:46085
I20260618 22:26:53.910228 12378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:46085 every 8 connection(s)
I20260618 22:26:53.911592 12302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:53.915026 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 12302
I20260618 22:26:53.915252 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:26:53.915654 12379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:53.916031 11211 external_mini_cluster.cc:1428] Setting key 2d30722be5baa097ff3b01f44f1b1ae7
I20260618 22:26:53.928951 12379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7: Bootstrap starting.
Jun 18 22:26:53 dist-test-slave-0956 krb5kdc[12289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:26:53.932682 12379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:53.933844 12379 log.cc:824] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:53.937063 12379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7: No bootstrap required, opened a new log
I20260618 22:26:53.939960 12382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:53.918256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37088 (local address 127.10.242.254:46085)
0618 22:26:53.919077 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:53.919089 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:53.919145 (+ 56us) server_negotiation.cc:407] Connection header received
0618 22:26:53.921437 (+ 2292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:53.921465 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:53.921881 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:53.922225 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:53.923883 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:53.925199 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:53.926883 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:53.927538 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:53.927659 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:53.932287 (+ 4628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:53.932295 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:53.932310 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:53.932349 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:53.935739 (+ 3390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:53.936560 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:53.936567 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:53.936574 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:53.936757 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:53.937201 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:53.937206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:53.937209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:53.937753 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:53.937954 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:53.938457 (+ 503us) server_negotiation.cc:299] Negotiation successful
0618 22:26:53.938920 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":405,"threads_started":1}
I20260618 22:26:53.944772 12379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69805a4c96284ab6af68e887cdd744f7" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 46085 } }
I20260618 22:26:53.945145 12379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:53.945235 12379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69805a4c96284ab6af68e887cdd744f7, State: Initialized, Role: FOLLOWER
I20260618 22:26:53.945740 12379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69805a4c96284ab6af68e887cdd744f7" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 46085 } }
I20260618 22:26:53.945964 12379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:53.946079 12379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:53.946216 12379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:53.948232 12379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69805a4c96284ab6af68e887cdd744f7" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 46085 } }
I20260618 22:26:53.948727 12379 leader_election.cc:302] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69805a4c96284ab6af68e887cdd744f7; no voters:
I20260618 22:26:53.949379 12379 leader_election.cc:288] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:53.949474 12384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:53.950080 12384 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7 [term 1 LEADER]: Becoming Leader. State: Replica: 69805a4c96284ab6af68e887cdd744f7, State: Running, Role: LEADER
I20260618 22:26:53.950644 12384 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69805a4c96284ab6af68e887cdd744f7" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 46085 } }
I20260618 22:26:53.951511 12379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:53.953464 12386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69805a4c96284ab6af68e887cdd744f7. Latest consensus state: current_term: 1 leader_uuid: "69805a4c96284ab6af68e887cdd744f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69805a4c96284ab6af68e887cdd744f7" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 46085 } } }
I20260618 22:26:53.953836 12386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:53.953903 12385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69805a4c96284ab6af68e887cdd744f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69805a4c96284ab6af68e887cdd744f7" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 46085 } } }
I20260618 22:26:53.954209 12385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:53.955909 12389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:53.960686 12389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:53.970803 12389 catalog_manager.cc:1347] Generated new cluster ID: 3761a92de20b450e820d8fa975624a42
I20260618 22:26:53.970925 12389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:53.997706 12389 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:53.999208 12389 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:54.013350 12389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7: Generated new TSK 0
I20260618 22:26:54.014330 12389 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
I20260618 22:26:54.069263 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:46085
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:41123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:54.385954 12405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:54.386232 12405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:54.386327 12405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:54.393600 12405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:54.393729 12405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:54.393816 12405 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:26:54.393894 12405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:26:54.403693 12405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:41123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.242.254:46085
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:26:54.404929 12405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:54.406278 12405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:54.419159 12410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:54.419134 12411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:54.419088 12413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:54.419945 12405 server_base.cc:1034] running on GCE node
I20260618 22:26:54.420550 12405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:54.421514 12405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:54.422761 12405 hybrid_clock.cc:648] HybridClock initialized: now 1781821614422689 us; error 43 us; skew 500 ppm
I20260618 22:26:54.423064 12405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:26:54 dist-test-slave-0956 krb5kdc[12289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821614, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:54.426235 12405 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:26:54.427613 12405 webserver.cc:460] Webserver started at http://127.10.242.193:37903/ using document root <none> and password file <none>
I20260618 22:26:54.428356 12405 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:54.428453 12405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:54.428761 12405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:54.431810 12405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "3881c132165a41b0bfef176cccc7ad45"
format_stamp: "Formatted at 2026-06-18 22:26:54 on dist-test-slave-0956"
server_key: "689c21de8a02452ace93f23611966124"
server_key_iv: "6822452b7b2916b7abe784cc578a3074"
server_key_version: "encryptionkey@0"
I20260618 22:26:54.432600 12405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "3881c132165a41b0bfef176cccc7ad45"
format_stamp: "Formatted at 2026-06-18 22:26:54 on dist-test-slave-0956"
server_key: "689c21de8a02452ace93f23611966124"
server_key_iv: "6822452b7b2916b7abe784cc578a3074"
server_key_version: "encryptionkey@0"
I20260618 22:26:54.437855 12405 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260618 22:26:54.441334 12420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:54.442656 12405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260618 22:26:54.442792 12405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "3881c132165a41b0bfef176cccc7ad45"
format_stamp: "Formatted at 2026-06-18 22:26:54 on dist-test-slave-0956"
server_key: "689c21de8a02452ace93f23611966124"
server_key_iv: "6822452b7b2916b7abe784cc578a3074"
server_key_version: "encryptionkey@0"
I20260618 22:26:54.442976 12405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:54.467785 12405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:54.472056 12405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:54.472317 12405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:54.474982 12405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:54.475075 12405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:54.475194 12405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:54.475253 12405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:26:54 dist-test-slave-0956 krb5kdc[12289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821614, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:26:54.498407 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:54.475473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:44623 (local address 127.10.242.254:46085)
0618 22:26:54.476097 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:54.476103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:54.477199 (+ 1096us) server_negotiation.cc:407] Connection header received
0618 22:26:54.479431 (+ 2232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:54.479438 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:54.479530 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:54.479613 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:54.482245 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:54.483419 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:54.485828 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:54.486627 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:54.486813 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:54.491947 (+ 5134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:54.491955 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:54.491960 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:54.492020 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:54.495100 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:54.495839 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:54.495845 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:54.495848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:54.495990 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:54.496421 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:54.496425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:54.496427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:54.496726 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:54.496854 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:54.497747 (+ 893us) server_negotiation.cc:299] Negotiation successful
0618 22:26:54.498003 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":400,"threads_started":1}
I20260618 22:26:54.499287 12430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:54.476165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:46085 (local address 127.10.242.193:44623)
0618 22:26:54.477002 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:54.477047 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:54.479068 (+ 2021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:54.480201 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:54.480210 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:54.481005 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:54.481998 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:54.482034 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:54.483626 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:54.483631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:54.485666 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:54.485682 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:54.486922 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:54.486934 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:54.487246 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:54.488145 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:54.488182 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:54.491649 (+ 3467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:54.495249 (+ 3600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:54.495259 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:54.495280 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:54.495706 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:54.496090 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:54.496095 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:54.496099 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:54.496309 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:54.496848 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:54.496856 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:54.497278 (+ 422us) client_negotiation.cc:769] Sending connection context
0618 22:26:54.497611 (+ 333us) client_negotiation.cc:240] Negotiation successful
0618 22:26:54.498090 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":327,"threads_started":1}
I20260618 22:26:54.522986 12405 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:40533
I20260618 22:26:54.523028 12540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:40533 every 8 connection(s)
I20260618 22:26:54.524503 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:54.511746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:46085 (local address 127.10.242.193:58175)
0618 22:26:54.512388 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:54.512411 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:54.512575 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:54.512909 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:54.512914 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:54.513583 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:54.513868 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:54.513878 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:54.514840 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:54.514845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:54.515859 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:54.515871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:54.516555 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:54.516561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:54.516711 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:54.517431 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:54.517454 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:54.518482 (+ 1028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:54.521964 (+ 3482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:54.521973 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:54.521977 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:54.522505 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:54.522929 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:54.522934 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:54.522937 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:54.523101 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:54.523637 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:54.523640 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:54.523810 (+ 170us) client_negotiation.cc:769] Sending connection context
0618 22:26:54.523927 (+ 117us) client_negotiation.cc:240] Negotiation successful
0618 22:26:54.524162 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":304,"threads_started":1}
I20260618 22:26:54.524554 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:54.511793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:58175 (local address 127.10.242.254:46085)
0618 22:26:54.511998 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:54.512005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:54.512501 (+ 496us) server_negotiation.cc:407] Connection header received
0618 22:26:54.512693 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:54.512696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:54.512761 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:54.512873 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:54.514005 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:54.514707 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:54.515990 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:54.516378 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:54.516483 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:54.518660 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:54.518667 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:54.518671 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:54.518714 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:54.521819 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:54.522628 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:54.522634 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:54.522637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:54.522811 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:54.523203 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:54.523208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:54.523211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:54.523526 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:54.523666 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:54.524053 (+ 387us) server_negotiation.cc:299] Negotiation successful
0618 22:26:54.524263 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260618 22:26:54.524595 12405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:26:54.525774 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 12405
I20260618 22:26:54.525998 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:26:54.526784 11211 external_mini_cluster.cc:1428] Setting key 42b60bf4a0286f00e4b9d81c3bbc4b0e
I20260618 22:26:54.527556 12541 heartbeater.cc:347] Connected to a master server at 127.10.242.254:46085
I20260618 22:26:54.527819 12541 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:54.529224 12541 heartbeater.cc:511] Master 127.10.242.254:46085 requested a full tablet report, sending...
I20260618 22:26:54.531219 12334 ts_manager.cc:194] Registered new tserver with Master: 3881c132165a41b0bfef176cccc7ad45 (127.10.242.193:40533)
I20260618 22:26:54.532662 12334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:44623
WARNING: no policy specified for kudu/127.10.242.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
I20260618 22:26:54.564898 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:46085
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:41123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:54.884060 12545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:54.884351 12545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:54.884446 12545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:54.891717 12545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:54.891829 12545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:54.891909 12545 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:26:54.891987 12545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:26:54.901887 12545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:41123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.242.254:46085
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:26:54.903170 12545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:54.904713 12545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:54.919529 12551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:54.919559 12550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:54.921984 12545 server_base.cc:1034] running on GCE node
W20260618 22:26:54.922837 12553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:54.923717 12545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:54.924678 12545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:54.925925 12545 hybrid_clock.cc:648] HybridClock initialized: now 1781821614925839 us; error 55 us; skew 500 ppm
I20260618 22:26:54.926254 12545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:26:54 dist-test-slave-0956 krb5kdc[12289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821614, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:54.929374 12545 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:26:54.931006 12545 webserver.cc:460] Webserver started at http://127.10.242.194:37249/ using document root <none> and password file <none>
I20260618 22:26:54.931679 12545 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:54.931764 12545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:54.932013 12545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:54.934867 12545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "ba2f52598a2b44e38b97ba0305df5b0c"
format_stamp: "Formatted at 2026-06-18 22:26:54 on dist-test-slave-0956"
server_key: "8329e892989d4b5187034d63acc7ddbb"
server_key_iv: "8b615defb63c2df80b043c9f089a5eae"
server_key_version: "encryptionkey@0"
I20260618 22:26:54.935626 12545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "ba2f52598a2b44e38b97ba0305df5b0c"
format_stamp: "Formatted at 2026-06-18 22:26:54 on dist-test-slave-0956"
server_key: "8329e892989d4b5187034d63acc7ddbb"
server_key_iv: "8b615defb63c2df80b043c9f089a5eae"
server_key_version: "encryptionkey@0"
I20260618 22:26:54.940934 12545 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260618 22:26:54.944734 12560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:54.946290 12545 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:26:54.946436 12545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "ba2f52598a2b44e38b97ba0305df5b0c"
format_stamp: "Formatted at 2026-06-18 22:26:54 on dist-test-slave-0956"
server_key: "8329e892989d4b5187034d63acc7ddbb"
server_key_iv: "8b615defb63c2df80b043c9f089a5eae"
server_key_version: "encryptionkey@0"
I20260618 22:26:54.946635 12545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:54.966521 12545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:54.970952 12545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:54.971182 12545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:54.973796 12545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:54.973874 12545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:54.973989 12545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:54.974051 12545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:26:54 dist-test-slave-0956 krb5kdc[12289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821614, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:26:55.002031 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:54.974275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:34491 (local address 127.10.242.254:46085)
0618 22:26:54.974477 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:54.974483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:54.975941 (+ 1458us) server_negotiation.cc:407] Connection header received
0618 22:26:54.978122 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:54.978129 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:54.978246 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:54.978331 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:54.980872 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:54.981889 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:54.987428 (+ 5539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:54.988103 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:54.988251 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:54.995063 (+ 6812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:54.995072 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:54.995077 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:54.995126 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:54.998037 (+ 2911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:54.999181 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:54.999188 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:54.999191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:54.999325 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:54.999904 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:54.999909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:54.999912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:55.000274 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:55.000398 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:55.001559 (+ 1161us) server_negotiation.cc:299] Negotiation successful
0618 22:26:55.001758 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260618 22:26:55.003876 12569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:54.974915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:46085 (local address 127.10.242.194:34491)
0618 22:26:54.975757 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:54.975803 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:54.977760 (+ 1957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:54.978984 (+ 1224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:54.978997 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:54.979783 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:54.980688 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:54.980708 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:54.982027 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:54.982032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:54.986760 (+ 4728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:54.986782 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:54.988418 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:54.988428 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:54.988820 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:54.990871 (+ 2051us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:54.990928 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:54.994716 (+ 3788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:54.998227 (+ 3511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:54.998239 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:54.998267 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:54.999013 (+ 746us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:54.999436 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:54.999442 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:54.999446 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:54.999759 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:55.000400 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:55.000411 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:55.001033 (+ 622us) client_negotiation.cc:769] Sending connection context
0618 22:26:55.001448 (+ 415us) client_negotiation.cc:240] Negotiation successful
0618 22:26:55.002073 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":353,"threads_started":1}
I20260618 22:26:55.022267 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.009117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:39615 (local address 127.10.242.254:46085)
0618 22:26:55.009433 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:55.009441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:55.009936 (+ 495us) server_negotiation.cc:407] Connection header received
0618 22:26:55.010124 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:55.010130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:55.010193 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:55.010272 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:55.011195 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.011923 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.013243 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.013800 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.013916 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.016199 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:55.016204 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:55.016207 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:55.016236 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:55.019084 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:55.020046 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:55.020051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:55.020055 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:55.020214 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:55.020607 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:55.020612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:55.020616 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:55.021041 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:55.021242 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:55.021718 (+ 476us) server_negotiation.cc:299] Negotiation successful
0618 22:26:55.021932 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196}
I20260618 22:26:55.022372 12631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.009067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:46085 (local address 127.10.242.194:39615)
0618 22:26:55.009839 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:55.009859 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:55.009991 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:55.010305 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:55.010309 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:55.010786 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:55.011067 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.011075 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.012045 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.012050 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.013054 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.013065 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.013993 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.014000 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.014214 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.014965 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:55.014987 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:55.015992 (+ 1005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:55.019212 (+ 3220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:55.019227 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:55.019232 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:55.019864 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:55.020367 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:55.020371 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:55.020373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:55.020495 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:55.021196 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:55.021201 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:55.021451 (+ 250us) client_negotiation.cc:769] Sending connection context
0618 22:26:55.021613 (+ 162us) client_negotiation.cc:240] Negotiation successful
0618 22:26:55.021949 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":480,"threads_started":1}
I20260618 22:26:55.030797 12545 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:45365
I20260618 22:26:55.030814 12679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:45365 every 8 connection(s)
I20260618 22:26:55.032048 12545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:26:55.033977 12680 heartbeater.cc:347] Connected to a master server at 127.10.242.254:46085
I20260618 22:26:55.034186 12680 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:55.034819 12680 heartbeater.cc:511] Master 127.10.242.254:46085 requested a full tablet report, sending...
I20260618 22:26:55.036065 12334 ts_manager.cc:194] Registered new tserver with Master: ba2f52598a2b44e38b97ba0305df5b0c (127.10.242.194:45365)
I20260618 22:26:55.036854 12334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:34491
I20260618 22:26:55.042294 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 12545
I20260618 22:26:55.042459 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:26:55.042824 11211 external_mini_cluster.cc:1428] Setting key a903c2b8b2b7617bad29674986edf791
WARNING: no policy specified for kudu/127.10.242.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
I20260618 22:26:55.079936 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:46085
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:41123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:55.397285 12684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:55.397583 12684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:55.397724 12684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:55.405236 12684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:55.405361 12684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:55.405486 12684 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:26:55.405597 12684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:26:55.415786 12684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:41123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.242.254:46085
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:26:55.417068 12684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:55.418391 12684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:55.431049 12689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:55.430989 12692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:55.430970 12690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:55.431766 12684 server_base.cc:1034] running on GCE node
I20260618 22:26:55.432395 12684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:55.433348 12684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:55.434608 12684 hybrid_clock.cc:648] HybridClock initialized: now 1781821615434519 us; error 43 us; skew 500 ppm
I20260618 22:26:55.434906 12684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:26:55 dist-test-slave-0956 krb5kdc[12289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821615, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:55.437922 12684 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:26:55.439384 12684 webserver.cc:460] Webserver started at http://127.10.242.195:38259/ using document root <none> and password file <none>
I20260618 22:26:55.440114 12684 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:55.440214 12684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:55.440517 12684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:55.443377 12684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "41a89fa29b614e2ea824d5df3ad6b6fa"
format_stamp: "Formatted at 2026-06-18 22:26:55 on dist-test-slave-0956"
server_key: "a56e4edf0d58950a39c3c621ad54b7e7"
server_key_iv: "834017912087258fecbde4217ceb97d6"
server_key_version: "encryptionkey@0"
I20260618 22:26:55.444201 12684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "41a89fa29b614e2ea824d5df3ad6b6fa"
format_stamp: "Formatted at 2026-06-18 22:26:55 on dist-test-slave-0956"
server_key: "a56e4edf0d58950a39c3c621ad54b7e7"
server_key_iv: "834017912087258fecbde4217ceb97d6"
server_key_version: "encryptionkey@0"
I20260618 22:26:55.449419 12684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260618 22:26:55.453131 12699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:55.454522 12684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:26:55.454674 12684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "41a89fa29b614e2ea824d5df3ad6b6fa"
format_stamp: "Formatted at 2026-06-18 22:26:55 on dist-test-slave-0956"
server_key: "a56e4edf0d58950a39c3c621ad54b7e7"
server_key_iv: "834017912087258fecbde4217ceb97d6"
server_key_version: "encryptionkey@0"
I20260618 22:26:55.454900 12684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:55.471094 12684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:55.475483 12684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:55.475754 12684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:55.478443 12684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:55.478528 12684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:55.478650 12684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:55.478767 12684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Jun 18 22:26:55 dist-test-slave-0956 krb5kdc[12289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821615, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:26:55.500965 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.478925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:34111 (local address 127.10.242.254:46085)
0618 22:26:55.479126 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:55.479134 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:55.480487 (+ 1353us) server_negotiation.cc:407] Connection header received
0618 22:26:55.482573 (+ 2086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:55.482580 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:55.482672 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:55.482782 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:55.484868 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.485909 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.488890 (+ 2981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.489447 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.489579 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.494088 (+ 4509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:55.494096 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:55.494101 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:55.494145 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:55.497601 (+ 3456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:55.498551 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:55.498557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:55.498562 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:55.498708 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:55.499143 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:55.499149 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:55.499152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:55.499448 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:55.499573 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:55.500394 (+ 821us) server_negotiation.cc:299] Negotiation successful
0618 22:26:55.500623 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:26:55.502115 12708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.479449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:46085 (local address 127.10.242.195:34111)
0618 22:26:55.480280 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:55.480339 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:55.482279 (+ 1940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:55.483038 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:55.483049 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:55.483788 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:55.484681 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.484709 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.486124 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.486130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.488714 (+ 2584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.488733 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.489737 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.489745 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.490086 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.490902 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:55.490924 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:55.493818 (+ 2894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:55.497769 (+ 3951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:55.497780 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:55.497798 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:55.498429 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:55.498823 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:55.498828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:55.498832 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:55.499033 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:55.499580 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:55.499588 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:55.499964 (+ 376us) client_negotiation.cc:769] Sending connection context
0618 22:26:55.500241 (+ 277us) client_negotiation.cc:240] Negotiation successful
0618 22:26:55.500847 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":350,"threads_started":1}
I20260618 22:26:55.526043 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.512463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:58843 (local address 127.10.242.254:46085)
0618 22:26:55.512654 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:55.512660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:55.512762 (+ 102us) server_negotiation.cc:407] Connection header received
0618 22:26:55.512954 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:55.512960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:55.513057 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:55.513200 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:55.514686 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.515531 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.517209 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.517750 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.517862 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.520432 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:55.520439 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:55.520444 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:55.520485 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:55.523572 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:55.524219 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:55.524225 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:55.524228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:55.524350 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:55.524677 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:55.524682 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:55.524686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:55.524948 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:55.525110 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:55.525496 (+ 386us) server_negotiation.cc:299] Negotiation successful
0618 22:26:55.525716 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:26:55.526176 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.511967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:46085 (local address 127.10.242.195:58843)
0618 22:26:55.512665 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:55.512685 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:55.512834 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:55.513383 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:55.513388 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:55.514022 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:55.514540 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.514552 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.515669 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.515678 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.517046 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.517063 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.517954 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.517961 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.518157 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.518925 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:55.518958 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:55.520187 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:55.523717 (+ 3530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:55.523722 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:55.523727 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:55.524100 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:55.524446 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:55.524451 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:55.524455 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:55.524576 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:55.525054 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:55.525059 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:55.525271 (+ 212us) client_negotiation.cc:769] Sending connection context
0618 22:26:55.525394 (+ 123us) client_negotiation.cc:240] Negotiation successful
0618 22:26:55.525742 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":541,"threads_started":1}
I20260618 22:26:55.531400 12684 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:34241
I20260618 22:26:55.531441 12818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:34241 every 8 connection(s)
I20260618 22:26:55.532532 12684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:26:55.534708 12819 heartbeater.cc:347] Connected to a master server at 127.10.242.254:46085
I20260618 22:26:55.534924 12819 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:55.535571 12819 heartbeater.cc:511] Master 127.10.242.254:46085 requested a full tablet report, sending...
I20260618 22:26:55.535890 12541 heartbeater.cc:503] Master 127.10.242.254:46085 was elected leader, sending a full tablet report...
I20260618 22:26:55.536542 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 12684
I20260618 22:26:55.536733 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:26:55.537129 12333 ts_manager.cc:194] Registered new tserver with Master: 41a89fa29b614e2ea824d5df3ad6b6fa (127.10.242.195:34241)
I20260618 22:26:55.537283 11211 external_mini_cluster.cc:1428] Setting key 8f4464f52772bf2013e9ec0b877e9dcd
I20260618 22:26:55.538209 12333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:34111
I20260618 22:26:55.540352 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 18 22:26:55 dist-test-slave-0956 krb5kdc[12289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821615, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 18 22:26:55 dist-test-slave-0956 krb5kdc[12289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821615, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:26:55.583635 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.570413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37090 (local address 127.10.242.254:46085)
0618 22:26:55.570568 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:55.570573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:55.570889 (+ 316us) server_negotiation.cc:407] Connection header received
0618 22:26:55.571034 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:55.571037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:55.571102 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:55.571194 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:55.572564 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.573393 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.574769 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.575170 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.575269 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.578927 (+ 3658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:55.578932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:55.578936 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:55.578972 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:55.581335 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:55.581929 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:55.581932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:55.581935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:55.582046 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:55.582378 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:55.582382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:55.582385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:55.582707 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:55.582899 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:55.583176 (+ 277us) server_negotiation.cc:299] Negotiation successful
0618 22:26:55.583342 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:26:55.587810 12333 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37090:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:26:55.594501 12333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:26:55.619576 12834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.608476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:40533 (local address 127.0.0.1:55990)
0618 22:26:55.609143 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:55.609164 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:55.609299 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:55.610439 (+ 1140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:55.610449 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:55.610484 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:55.610942 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.610954 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.613224 (+ 2270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.613230 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.615849 (+ 2619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.615868 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.618457 (+ 2589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.618464 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.618731 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.618799 (+ 68us) client_negotiation.cc:769] Sending connection context
0618 22:26:55.618922 (+ 123us) client_negotiation.cc:240] Negotiation successful
0618 22:26:55.619152 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":1032,"threads_started":1}
I20260618 22:26:55.621410 12835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.608704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55990 (local address 127.10.242.193:40533)
0618 22:26:55.609742 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:55.609749 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:55.609771 (+ 22us) server_negotiation.cc:407] Connection header received
0618 22:26:55.609867 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:55.609874 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:55.610136 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:55.611185 (+ 1049us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:55.611403 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.613004 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.616483 (+ 3479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.618172 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.620521 (+ 2349us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.620750 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:55.620840 (+ 90us) server_negotiation.cc:299] Negotiation successful
0618 22:26:55.621029 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":845,"thread_start_us":726,"threads_started":1}
I20260618 22:26:55.626181 12471 tablet_service.cc:1467] Processing CreateTablet for tablet e9ba3dfb0185439ba5c2e161bb6ecb54 (DEFAULT_TABLE table=test-table [id=732f7ae97e9f4a07891c18ce7ba71a96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:55.627681 12471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9ba3dfb0185439ba5c2e161bb6ecb54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:55.628962 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.610114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49594 (local address 127.10.242.194:45365)
0618 22:26:55.622535 (+ 12421us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:55.622544 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:55.622576 (+ 32us) server_negotiation.cc:407] Connection header received
0618 22:26:55.622677 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:55.622685 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:55.622973 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:55.623181 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:55.623541 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.624426 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.626623 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.627954 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.628107 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.628260 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:55.628549 (+ 289us) server_negotiation.cc:299] Negotiation successful
0618 22:26:55.628711 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12259,"thread_start_us":13211,"threads_started":1}
I20260618 22:26:55.628984 12834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.609907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:45365 (local address 127.0.0.1:49594)
0618 22:26:55.619872 (+ 9965us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:55.619894 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:55.620029 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:55.623178 (+ 3149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:55.623183 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:55.623213 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:55.623397 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.623409 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.624569 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.624574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.626440 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.626455 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.628100 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.628106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.628337 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.628386 (+ 49us) client_negotiation.cc:769] Sending connection context
0618 22:26:55.628489 (+ 103us) client_negotiation.cc:240] Negotiation successful
0618 22:26:55.628631 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9852,"mutex_wait_us":19,"thread_start_us":12187,"threads_started":1}
I20260618 22:26:55.632293 12832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.607108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:34241 (local address 127.0.0.1:50014)
0618 22:26:55.607886 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:55.607917 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:55.608051 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:55.625694 (+ 17643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:55.625699 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:55.625727 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:55.626143 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.626156 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.627794 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.627799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.629744 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.629758 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.631504 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.631510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.631672 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.631705 (+ 33us) client_negotiation.cc:769] Sending connection context
0618 22:26:55.631801 (+ 96us) client_negotiation.cc:240] Negotiation successful
0618 22:26:55.631936 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":46,"thread_start_us":425,"threads_started":1}
I20260618 22:26:55.632311 12833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.607332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50014 (local address 127.10.242.195:34241)
0618 22:26:55.623074 (+ 15742us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:55.623080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:55.623106 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:26:55.623208 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:55.623215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:55.625488 (+ 2273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:55.626347 (+ 859us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:55.626648 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.627663 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.629922 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.631298 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.631480 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.631636 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:55.631849 (+ 213us) server_negotiation.cc:299] Negotiation successful
0618 22:26:55.632002 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15570,"thread_start_us":15397,"threads_started":1}
I20260618 22:26:55.632395 12610 tablet_service.cc:1467] Processing CreateTablet for tablet e9ba3dfb0185439ba5c2e161bb6ecb54 (DEFAULT_TABLE table=test-table [id=732f7ae97e9f4a07891c18ce7ba71a96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:55.633942 12610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9ba3dfb0185439ba5c2e161bb6ecb54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:55.636093 12749 tablet_service.cc:1467] Processing CreateTablet for tablet e9ba3dfb0185439ba5c2e161bb6ecb54 (DEFAULT_TABLE table=test-table [id=732f7ae97e9f4a07891c18ce7ba71a96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:55.637642 12749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9ba3dfb0185439ba5c2e161bb6ecb54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:55.642657 12838 tablet_bootstrap.cc:492] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45: Bootstrap starting.
I20260618 22:26:55.645550 12839 tablet_bootstrap.cc:492] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c: Bootstrap starting.
I20260618 22:26:55.646642 12838 tablet_bootstrap.cc:654] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:55.647727 12838 log.cc:824] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:55.648996 12839 tablet_bootstrap.cc:654] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:55.650197 12839 log.cc:824] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:55.656140 12839 tablet_bootstrap.cc:492] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c: No bootstrap required, opened a new log
I20260618 22:26:55.656178 12838 tablet_bootstrap.cc:492] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45: No bootstrap required, opened a new log
I20260618 22:26:55.656457 12839 ts_tablet_manager.cc:1397] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260618 22:26:55.656458 12838 ts_tablet_manager.cc:1397] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45: Time spent bootstrapping tablet: real 0.014s user 0.013s sys 0.000s
I20260618 22:26:55.656929 12842 tablet_bootstrap.cc:492] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa: Bootstrap starting.
I20260618 22:26:55.660661 12842 tablet_bootstrap.cc:654] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:55.662546 12842 log.cc:824] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:55.665074 12839 raft_consensus.cc:348] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } }
I20260618 22:26:55.665485 12839 raft_consensus.cc:374] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:55.665575 12839 raft_consensus.cc:729] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2f52598a2b44e38b97ba0305df5b0c, State: Initialized, Role: FOLLOWER
I20260618 22:26:55.665910 12842 tablet_bootstrap.cc:492] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa: No bootstrap required, opened a new log
I20260618 22:26:55.666185 12842 ts_tablet_manager.cc:1397] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260618 22:26:55.666167 12839 consensus_queue.cc:260] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } }
I20260618 22:26:55.666514 12838 raft_consensus.cc:348] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } }
I20260618 22:26:55.666927 12838 raft_consensus.cc:374] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:55.667032 12838 raft_consensus.cc:729] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3881c132165a41b0bfef176cccc7ad45, State: Initialized, Role: FOLLOWER
I20260618 22:26:55.667388 12680 heartbeater.cc:503] Master 127.10.242.254:46085 was elected leader, sending a full tablet report...
I20260618 22:26:55.667812 12839 ts_tablet_manager.cc:1428] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c: Time spent starting tablet: real 0.011s user 0.009s sys 0.001s
I20260618 22:26:55.667728 12838 consensus_queue.cc:260] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } }
I20260618 22:26:55.669940 12838 ts_tablet_manager.cc:1428] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260618 22:26:55.676126 12842 raft_consensus.cc:348] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } }
I20260618 22:26:55.676487 12842 raft_consensus.cc:374] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:55.676559 12842 raft_consensus.cc:729] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41a89fa29b614e2ea824d5df3ad6b6fa, State: Initialized, Role: FOLLOWER
I20260618 22:26:55.677049 12842 consensus_queue.cc:260] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } }
I20260618 22:26:55.678051 12819 heartbeater.cc:503] Master 127.10.242.254:46085 was elected leader, sending a full tablet report...
I20260618 22:26:55.678613 12842 ts_tablet_manager.cc:1428] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
W20260618 22:26:55.779700 12542 tablet.cc:2369] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:26:55.783936 12681 tablet.cc:2369] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:26:55.784233 12820 tablet.cc:2369] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:26:55.813859 12844 raft_consensus.cc:482] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:26:55.814170 12844 raft_consensus.cc:504] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } }
I20260618 22:26:55.815822 12844 leader_election.cc:288] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41a89fa29b614e2ea824d5df3ad6b6fa (127.10.242.195:34241), 3881c132165a41b0bfef176cccc7ad45 (127.10.242.193:40533)
I20260618 22:26:55.823845 12833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.816343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:46249 (local address 127.10.242.195:34241)
0618 22:26:55.816588 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:55.816594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:55.817007 (+ 413us) server_negotiation.cc:407] Connection header received
0618 22:26:55.817323 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:55.817329 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:55.817406 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:55.817488 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:55.818400 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.819450 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.821871 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.823025 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.823208 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.823284 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:55.823532 (+ 248us) server_negotiation.cc:299] Negotiation successful
0618 22:26:55.823625 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260618 22:26:55.824030 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.816242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:34241 (local address 127.10.242.194:46249)
0618 22:26:55.816900 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:55.816924 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:55.817130 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:55.817737 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:55.817742 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:55.817776 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:55.818253 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.818265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.819578 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.819581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.821694 (+ 2113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.821710 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.823238 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.823242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.823359 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.823394 (+ 35us) client_negotiation.cc:769] Sending connection context
0618 22:26:55.823490 (+ 96us) client_negotiation.cc:240] Negotiation successful
0618 22:26:55.823632 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":27,"thread_start_us":314,"threads_started":1}
I20260618 22:26:55.824174 12835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.817240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:53091 (local address 127.10.242.193:40533)
0618 22:26:55.817431 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:55.817436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:55.818223 (+ 787us) server_negotiation.cc:407] Connection header received
0618 22:26:55.818390 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:55.818395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:55.818471 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:55.818579 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:55.819043 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.819879 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.822138 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.823314 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.823435 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.823493 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:55.823833 (+ 340us) server_negotiation.cc:299] Negotiation successful
0618 22:26:55.823954 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:26:55.824692 12848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.816929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:40533 (local address 127.10.242.194:53091)
0618 22:26:55.818096 (+ 1167us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:55.818113 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:55.818244 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:55.818583 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:55.818586 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:55.818603 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:55.818887 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.818894 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.820000 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.820005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.821980 (+ 1975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.821989 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.823514 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.823519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.823674 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.823703 (+ 29us) client_negotiation.cc:769] Sending connection context
0618 22:26:55.823793 (+ 90us) client_negotiation.cc:240] Negotiation successful
0618 22:26:55.823913 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1023,"mutex_wait_us":1,"thread_start_us":316,"threads_started":1}
I20260618 22:26:55.825103 12773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ba3dfb0185439ba5c2e161bb6ecb54" candidate_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" is_pre_election: true
I20260618 22:26:55.825639 12773 raft_consensus.cc:2454] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2f52598a2b44e38b97ba0305df5b0c in term 0.
I20260618 22:26:55.825662 12491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ba3dfb0185439ba5c2e161bb6ecb54" candidate_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3881c132165a41b0bfef176cccc7ad45" is_pre_election: true
I20260618 22:26:55.826131 12491 raft_consensus.cc:2454] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2f52598a2b44e38b97ba0305df5b0c in term 0.
I20260618 22:26:55.826282 12564 leader_election.cc:302] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41a89fa29b614e2ea824d5df3ad6b6fa, ba2f52598a2b44e38b97ba0305df5b0c; no voters:
I20260618 22:26:55.826650 12844 raft_consensus.cc:2781] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:26:55.826774 12844 raft_consensus.cc:482] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:26:55.826910 12844 raft_consensus.cc:3037] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:55.829453 12844 raft_consensus.cc:504] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } }
I20260618 22:26:55.830025 12844 leader_election.cc:288] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [CANDIDATE]: Term 1 election: Requested vote from peers 41a89fa29b614e2ea824d5df3ad6b6fa (127.10.242.195:34241), 3881c132165a41b0bfef176cccc7ad45 (127.10.242.193:40533)
I20260618 22:26:55.830592 12773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ba3dfb0185439ba5c2e161bb6ecb54" candidate_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa"
I20260618 22:26:55.830611 12491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ba3dfb0185439ba5c2e161bb6ecb54" candidate_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3881c132165a41b0bfef176cccc7ad45"
I20260618 22:26:55.830775 12491 raft_consensus.cc:3037] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:55.830799 12773 raft_consensus.cc:3037] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:55.832926 12491 raft_consensus.cc:2454] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2f52598a2b44e38b97ba0305df5b0c in term 1.
I20260618 22:26:55.832928 12773 raft_consensus.cc:2454] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2f52598a2b44e38b97ba0305df5b0c in term 1.
I20260618 22:26:55.833349 12564 leader_election.cc:302] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3881c132165a41b0bfef176cccc7ad45, ba2f52598a2b44e38b97ba0305df5b0c; no voters:
I20260618 22:26:55.833653 12844 raft_consensus.cc:2781] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:55.834126 12844 raft_consensus.cc:686] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [term 1 LEADER]: Becoming Leader. State: Replica: ba2f52598a2b44e38b97ba0305df5b0c, State: Running, Role: LEADER
I20260618 22:26:55.834583 12844 consensus_queue.cc:237] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } }
I20260618 22:26:55.839442 12333 catalog_manager.cc:5495] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c reported cstate change: term changed from 0 to 1, leader changed from <none> to ba2f52598a2b44e38b97ba0305df5b0c (127.10.242.194). New cstate: current_term: 1 leader_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } health_report { overall_health: HEALTHY } } }
I20260618 22:26:55.867151 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.861192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44894 (local address 127.10.242.194:45365)
0618 22:26:55.861395 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:55.861401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:55.861472 (+ 71us) server_negotiation.cc:407] Connection header received
0618 22:26:55.861830 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:55.861834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:55.861910 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:55.862038 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:55.862443 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.863154 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.864527 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.865366 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.865492 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.865984 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:55.866544 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:55.866677 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:55.866804 (+ 127us) server_negotiation.cc:299] Negotiation successful
0618 22:26:55.866886 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260618 22:26:55.875586 12491 raft_consensus.cc:1261] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45 [term 1 FOLLOWER]: Refusing update from remote peer ba2f52598a2b44e38b97ba0305df5b0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:55.875679 12773 raft_consensus.cc:1261] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 1 FOLLOWER]: Refusing update from remote peer ba2f52598a2b44e38b97ba0305df5b0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:55.876569 12849 consensus_queue.cc:1035] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:55.876930 12844 consensus_queue.cc:1035] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:55.887758 12852 mvcc.cc:205] Tried to move back new op lower bound from 7298341338616594432 to 7298341338462031872. Current Snapshot: MvccSnapshot[applied={T|T < 7298341338616594432}]
I20260618 22:26:55.908608 12333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:26:55.908862 12333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:26:55.912973 12333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7: Sending DeleteTablet for 3 replicas of tablet e9ba3dfb0185439ba5c2e161bb6ecb54
I20260618 22:26:55.914836 12749 tablet_service.cc:1514] Processing DeleteTablet for tablet e9ba3dfb0185439ba5c2e161bb6ecb54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:55 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:50014
I20260618 22:26:55.914799 12471 tablet_service.cc:1514] Processing DeleteTablet for tablet e9ba3dfb0185439ba5c2e161bb6ecb54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:55 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:55990
I20260618 22:26:55.915995 12610 tablet_service.cc:1514] Processing DeleteTablet for tablet e9ba3dfb0185439ba5c2e161bb6ecb54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:55 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:49594
I20260618 22:26:55.922583 12333 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37090:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:26:55.923429 12333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:26:55.929580 12864 tablet_replica.cc:330] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c: stopping tablet replica
I20260618 22:26:55.930048 12864 raft_consensus.cc:2229] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:26:55.930691 12864 raft_consensus.cc:2258] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:26:55.937630 12470 tablet_service.cc:1467] Processing CreateTablet for tablet 220568d04c1f4f5796092c833adcb471 (DEFAULT_TABLE table=test-table [id=5cdd6c5eb16f4478a8eb9c8b8608aa89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:55.938290 12470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 220568d04c1f4f5796092c833adcb471. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:55.938968 12748 tablet_service.cc:1467] Processing CreateTablet for tablet 220568d04c1f4f5796092c833adcb471 (DEFAULT_TABLE table=test-table [id=5cdd6c5eb16f4478a8eb9c8b8608aa89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:55.939481 12748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 220568d04c1f4f5796092c833adcb471. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:55.944687 12862 tablet_replica.cc:330] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa: stopping tablet replica
I20260618 22:26:55.946297 12838 tablet_bootstrap.cc:492] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45: Bootstrap starting.
I20260618 22:26:55.946947 12842 tablet_bootstrap.cc:492] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa: Bootstrap starting.
I20260618 22:26:55.947860 12838 tablet_bootstrap.cc:654] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:55.947877 12864 ts_tablet_manager.cc:1905] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:26:55.948447 12842 tablet_bootstrap.cc:654] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:55.950085 12610 tablet_service.cc:1467] Processing CreateTablet for tablet 220568d04c1f4f5796092c833adcb471 (DEFAULT_TABLE table=test-table [id=5cdd6c5eb16f4478a8eb9c8b8608aa89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:55.950562 12610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 220568d04c1f4f5796092c833adcb471. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:55.951411 12862 raft_consensus.cc:2229] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:26:55.952001 12862 raft_consensus.cc:2258] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:26:55.954438 12862 ts_tablet_manager.cc:1905] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:26:55.956990 12864 ts_tablet_manager.cc:1918] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:26:55.958760 12842 tablet_bootstrap.cc:492] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa: No bootstrap required, opened a new log
I20260618 22:26:55.958993 12842 ts_tablet_manager.cc:1397] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260618 22:26:55.959973 12842 raft_consensus.cc:348] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } }
I20260618 22:26:55.960305 12842 raft_consensus.cc:374] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:55.960449 12842 raft_consensus.cc:729] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41a89fa29b614e2ea824d5df3ad6b6fa, State: Initialized, Role: FOLLOWER
I20260618 22:26:55.960881 12838 tablet_bootstrap.cc:492] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45: No bootstrap required, opened a new log
I20260618 22:26:55.960793 12842 consensus_queue.cc:260] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } }
I20260618 22:26:55.961030 12838 ts_tablet_manager.cc:1397] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.004s
I20260618 22:26:55.961823 12838 raft_consensus.cc:348] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } }
I20260618 22:26:55.962037 12838 raft_consensus.cc:374] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:55.962124 12838 raft_consensus.cc:729] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3881c132165a41b0bfef176cccc7ad45, State: Initialized, Role: FOLLOWER
I20260618 22:26:55.962477 12838 consensus_queue.cc:260] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } }
I20260618 22:26:55.963225 12838 ts_tablet_manager.cc:1428] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 22:26:55.964087 12842 ts_tablet_manager.cc:1428] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260618 22:26:55.964123 12863 tablet_replica.cc:330] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45: stopping tablet replica
I20260618 22:26:55.964550 12863 raft_consensus.cc:2229] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:26:55.965034 12863 raft_consensus.cc:2258] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:26:55.971433 12863 ts_tablet_manager.cc:1905] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:26:55.973126 12862 ts_tablet_manager.cc:1918] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:26:55.964365 12839 tablet_bootstrap.cc:492] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c: Bootstrap starting.
I20260618 22:26:55.973308 12862 log.cc:1196] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/e9ba3dfb0185439ba5c2e161bb6ecb54
I20260618 22:26:55.973497 12864 log.cc:1196] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/e9ba3dfb0185439ba5c2e161bb6ecb54
I20260618 22:26:55.973796 12862 ts_tablet_manager.cc:1939] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 41a89fa29b614e2ea824d5df3ad6b6fa: Deleting consensus metadata
I20260618 22:26:55.974843 12864 ts_tablet_manager.cc:1939] T e9ba3dfb0185439ba5c2e161bb6ecb54 P ba2f52598a2b44e38b97ba0305df5b0c: Deleting consensus metadata
I20260618 22:26:55.975029 12845 raft_consensus.cc:482] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:26:55.975186 12845 raft_consensus.cc:504] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } }
I20260618 22:26:55.975487 12839 tablet_bootstrap.cc:654] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:55.975678 12321 catalog_manager.cc:4841] TS 41a89fa29b614e2ea824d5df3ad6b6fa (127.10.242.195:34241): tablet e9ba3dfb0185439ba5c2e161bb6ecb54 (table test-table [id=732f7ae97e9f4a07891c18ce7ba71a96]) successfully deleted
I20260618 22:26:55.976961 12845 leader_election.cc:288] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2f52598a2b44e38b97ba0305df5b0c (127.10.242.194:45365), 41a89fa29b614e2ea824d5df3ad6b6fa (127.10.242.195:34241)
I20260618 22:26:55.977757 12839 tablet_bootstrap.cc:492] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c: No bootstrap required, opened a new log
I20260618 22:26:55.977908 12839 ts_tablet_manager.cc:1397] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.001s
I20260618 22:26:55.978682 12839 raft_consensus.cc:348] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } }
I20260618 22:26:55.978925 12839 raft_consensus.cc:374] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:55.979018 12839 raft_consensus.cc:729] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2f52598a2b44e38b97ba0305df5b0c, State: Initialized, Role: FOLLOWER
I20260618 22:26:55.979312 12839 consensus_queue.cc:260] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } }
I20260618 22:26:55.981020 12863 ts_tablet_manager.cc:1918] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:26:55.981241 12863 log.cc:1196] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/e9ba3dfb0185439ba5c2e161bb6ecb54
I20260618 22:26:55.981797 12863 ts_tablet_manager.cc:1939] T e9ba3dfb0185439ba5c2e161bb6ecb54 P 3881c132165a41b0bfef176cccc7ad45: Deleting consensus metadata
I20260618 22:26:55.982457 12839 ts_tablet_manager.cc:1428] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260618 22:26:55.984443 12321 catalog_manager.cc:4841] TS 3881c132165a41b0bfef176cccc7ad45 (127.10.242.193:40533): tablet e9ba3dfb0185439ba5c2e161bb6ecb54 (table test-table [id=732f7ae97e9f4a07891c18ce7ba71a96]) successfully deleted
I20260618 22:26:55.984964 12321 catalog_manager.cc:4841] TS ba2f52598a2b44e38b97ba0305df5b0c (127.10.242.194:45365): tablet e9ba3dfb0185439ba5c2e161bb6ecb54 (table test-table [id=732f7ae97e9f4a07891c18ce7ba71a96]) successfully deleted
I20260618 22:26:55.991930 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.977777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:44287 (local address 127.10.242.194:45365)
0618 22:26:55.977941 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:55.977946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:55.985415 (+ 7469us) server_negotiation.cc:407] Connection header received
0618 22:26:55.985692 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:55.985697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:55.985784 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:55.985844 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:55.986612 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.987637 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.989741 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.990944 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.991088 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.991162 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:55.991494 (+ 332us) server_negotiation.cc:299] Negotiation successful
0618 22:26:55.991632 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:26:55.992110 12868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.978671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:45365 (local address 127.10.242.193:44287)
0618 22:26:55.985320 (+ 6649us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:55.985341 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:55.985523 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:55.986049 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:55.986054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:55.986085 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:55.986457 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.986472 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.987759 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.987765 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.989598 (+ 1833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.989610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.991150 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.991156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.991325 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.991360 (+ 35us) client_negotiation.cc:769] Sending connection context
0618 22:26:55.991456 (+ 96us) client_negotiation.cc:240] Negotiation successful
0618 22:26:55.991596 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6515,"thread_start_us":6329,"threads_started":1}
I20260618 22:26:55.993147 12833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.985624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:45173 (local address 127.10.242.195:34241)
0618 22:26:55.985817 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:55.985823 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:55.986328 (+ 505us) server_negotiation.cc:407] Connection header received
0618 22:26:55.986476 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:55.986481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:55.986557 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:55.986639 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:55.987364 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.988528 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.990914 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.991945 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.992079 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.992143 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:55.992736 (+ 593us) server_negotiation.cc:299] Negotiation successful
0618 22:26:55.992861 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260618 22:26:55.993137 12635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220568d04c1f4f5796092c833adcb471" candidate_uuid: "3881c132165a41b0bfef176cccc7ad45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" is_pre_election: true
I20260618 22:26:55.993610 12635 raft_consensus.cc:2454] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3881c132165a41b0bfef176cccc7ad45 in term 0.
I20260618 22:26:55.994225 12424 leader_election.cc:302] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3881c132165a41b0bfef176cccc7ad45, ba2f52598a2b44e38b97ba0305df5b0c; no voters:
I20260618 22:26:55.994417 12869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:55.985449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:34241 (local address 127.10.242.193:45173)
0618 22:26:55.986225 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:55.986239 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:55.986350 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:55.986796 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:55.986801 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:55.986823 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:55.987164 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.987176 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.988704 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.988709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.990745 (+ 2036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:55.990760 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:55.992464 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:55.992471 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:55.992589 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:55.992612 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:26:55.992689 (+ 77us) client_negotiation.cc:240] Negotiation successful
0618 22:26:55.994033 (+ 1344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":493,"threads_started":1}
I20260618 22:26:55.994624 12845 raft_consensus.cc:2781] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:26:55.994732 12845 raft_consensus.cc:482] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:26:55.994807 12845 raft_consensus.cc:3037] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:55.995160 12773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220568d04c1f4f5796092c833adcb471" candidate_uuid: "3881c132165a41b0bfef176cccc7ad45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" is_pre_election: true
I20260618 22:26:55.995394 12773 raft_consensus.cc:2454] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3881c132165a41b0bfef176cccc7ad45 in term 0.
I20260618 22:26:55.997269 12845 raft_consensus.cc:504] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } }
I20260618 22:26:55.997831 12845 leader_election.cc:288] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [CANDIDATE]: Term 1 election: Requested vote from peers ba2f52598a2b44e38b97ba0305df5b0c (127.10.242.194:45365), 41a89fa29b614e2ea824d5df3ad6b6fa (127.10.242.195:34241)
I20260618 22:26:55.998306 12635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220568d04c1f4f5796092c833adcb471" candidate_uuid: "3881c132165a41b0bfef176cccc7ad45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2f52598a2b44e38b97ba0305df5b0c"
I20260618 22:26:55.998322 12773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220568d04c1f4f5796092c833adcb471" candidate_uuid: "3881c132165a41b0bfef176cccc7ad45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa"
I20260618 22:26:55.998512 12773 raft_consensus.cc:3037] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:55.998520 12635 raft_consensus.cc:3037] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:56.001332 12773 raft_consensus.cc:2454] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3881c132165a41b0bfef176cccc7ad45 in term 1.
I20260618 22:26:56.001798 12424 leader_election.cc:302] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3881c132165a41b0bfef176cccc7ad45, 41a89fa29b614e2ea824d5df3ad6b6fa; no voters:
I20260618 22:26:56.002228 12845 raft_consensus.cc:2781] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:56.002754 12635 raft_consensus.cc:2454] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3881c132165a41b0bfef176cccc7ad45 in term 1.
I20260618 22:26:56.002825 12845 raft_consensus.cc:686] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [term 1 LEADER]: Becoming Leader. State: Replica: 3881c132165a41b0bfef176cccc7ad45, State: Running, Role: LEADER
I20260618 22:26:56.003235 12845 consensus_queue.cc:237] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } }
I20260618 22:26:56.007058 12332 catalog_manager.cc:5495] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3881c132165a41b0bfef176cccc7ad45 (127.10.242.193). New cstate: current_term: 1 leader_uuid: "3881c132165a41b0bfef176cccc7ad45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } health_report { overall_health: UNKNOWN } } }
Jun 18 22:26:56 dist-test-slave-0956 krb5kdc[12289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821613, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:26:56.064774 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:56.048066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47380 (local address 127.10.242.254:46085)
0618 22:26:56.048253 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:56.048262 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:56.048873 (+ 611us) server_negotiation.cc:407] Connection header received
0618 22:26:56.049049 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:56.049054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:56.049175 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:56.049277 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:56.050779 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.051960 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.053927 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.054555 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.054678 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:56.059119 (+ 4441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:56.059125 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:56.059128 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:56.059161 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:56.061751 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:56.062598 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:56.062603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:56.062605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:56.062803 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:56.063179 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:56.063183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:56.063185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:56.063505 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:56.063612 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:56.064163 (+ 551us) server_negotiation.cc:299] Negotiation successful
0618 22:26:56.064464 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":68}
I20260618 22:26:56.064777 12877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:56.048098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:46085 (local address 127.0.0.1:47380)
0618 22:26:56.048752 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:56.048772 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:56.048927 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:56.049316 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:56.049321 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:56.050222 (+ 901us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:56.050600 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:56.050612 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.052110 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.052116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:56.053784 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:56.053798 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.054750 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.054757 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:56.054933 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:56.056112 (+ 1179us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:56.056136 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:56.058918 (+ 2782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:56.061873 (+ 2955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:56.061886 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:56.061895 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:56.062470 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:56.062907 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:56.062912 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:56.062916 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:56.063067 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:56.063623 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:56.063632 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:56.063843 (+ 211us) client_negotiation.cc:769] Sending connection context
0618 22:26:56.064023 (+ 180us) client_negotiation.cc:240] Negotiation successful
0618 22:26:56.064319 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":348,"threads_started":1}
I20260618 22:26:56.070958 12332 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:47380:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260618 22:26:56.072244 12332 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:26:56.078822 12471 tablet_service.cc:1467] Processing CreateTablet for tablet a32c6efbb81946eea0252297bfbe93a5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8185c99534b646e689624c2ad4ffed60]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:26:56.079113 12748 tablet_service.cc:1467] Processing CreateTablet for tablet a32c6efbb81946eea0252297bfbe93a5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8185c99534b646e689624c2ad4ffed60]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:26:56.079392 12471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a32c6efbb81946eea0252297bfbe93a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:56.079587 12748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a32c6efbb81946eea0252297bfbe93a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:56.084333 12842 tablet_bootstrap.cc:492] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa: Bootstrap starting.
I20260618 22:26:56.085623 12610 tablet_service.cc:1467] Processing CreateTablet for tablet a32c6efbb81946eea0252297bfbe93a5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8185c99534b646e689624c2ad4ffed60]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:26:56.085871 12842 tablet_bootstrap.cc:654] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:56.086194 12610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a32c6efbb81946eea0252297bfbe93a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:56.087468 12842 tablet_bootstrap.cc:492] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa: No bootstrap required, opened a new log
I20260618 22:26:56.087618 12842 ts_tablet_manager.cc:1397] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260618 22:26:56.088163 12842 raft_consensus.cc:348] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } }
I20260618 22:26:56.088394 12842 raft_consensus.cc:374] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:56.088469 12842 raft_consensus.cc:729] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41a89fa29b614e2ea824d5df3ad6b6fa, State: Initialized, Role: FOLLOWER
I20260618 22:26:56.088722 12842 consensus_queue.cc:260] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } }
I20260618 22:26:56.089284 12842 ts_tablet_manager.cc:1428] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 22:26:56.089205 12846 tablet_replica.cc:439] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } }
I20260618 22:26:56.089531 12846 tablet_replica.cc:442] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:26:56.091022 12838 tablet_bootstrap.cc:492] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45: Bootstrap starting.
I20260618 22:26:56.092635 12838 tablet_bootstrap.cc:654] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:56.094892 12838 tablet_bootstrap.cc:492] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45: No bootstrap required, opened a new log
I20260618 22:26:56.095044 12838 ts_tablet_manager.cc:1397] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260618 22:26:56.095443 12839 tablet_bootstrap.cc:492] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c: Bootstrap starting.
I20260618 22:26:56.095721 12838 raft_consensus.cc:348] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } }
I20260618 22:26:56.095942 12838 raft_consensus.cc:374] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:56.096007 12838 raft_consensus.cc:729] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3881c132165a41b0bfef176cccc7ad45, State: Initialized, Role: FOLLOWER
I20260618 22:26:56.096251 12838 consensus_queue.cc:260] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } }
I20260618 22:26:56.096837 12838 ts_tablet_manager.cc:1428] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 22:26:56.096714 12845 tablet_replica.cc:439] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } }
I20260618 22:26:56.097009 12845 tablet_replica.cc:442] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:26:56.097246 12839 tablet_bootstrap.cc:654] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:56.099743 12839 tablet_bootstrap.cc:492] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c: No bootstrap required, opened a new log
I20260618 22:26:56.099910 12839 ts_tablet_manager.cc:1397] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260618 22:26:56.100536 12839 raft_consensus.cc:348] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } }
I20260618 22:26:56.100800 12839 raft_consensus.cc:374] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:56.100891 12839 raft_consensus.cc:729] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2f52598a2b44e38b97ba0305df5b0c, State: Initialized, Role: FOLLOWER
I20260618 22:26:56.101142 12839 consensus_queue.cc:260] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } }
I20260618 22:26:56.101693 12839 ts_tablet_manager.cc:1428] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 22:26:56.101528 12859 tablet_replica.cc:439] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } }
I20260618 22:26:56.102512 12859 tablet_replica.cc:442] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:26:56.134454 12859 raft_consensus.cc:482] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:26:56.134687 12859 raft_consensus.cc:504] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } }
I20260618 22:26:56.135404 12859 leader_election.cc:288] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3881c132165a41b0bfef176cccc7ad45 (127.10.242.193:40533), 41a89fa29b614e2ea824d5df3ad6b6fa (127.10.242.195:34241)
I20260618 22:26:56.136003 12773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a32c6efbb81946eea0252297bfbe93a5" candidate_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" is_pre_election: true
I20260618 22:26:56.136026 12491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a32c6efbb81946eea0252297bfbe93a5" candidate_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3881c132165a41b0bfef176cccc7ad45" is_pre_election: true
I20260618 22:26:56.136248 12773 raft_consensus.cc:2454] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2f52598a2b44e38b97ba0305df5b0c in term 0.
I20260618 22:26:56.136289 12491 raft_consensus.cc:2454] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2f52598a2b44e38b97ba0305df5b0c in term 0.
I20260618 22:26:56.136708 12564 leader_election.cc:302] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3881c132165a41b0bfef176cccc7ad45, ba2f52598a2b44e38b97ba0305df5b0c; no voters:
I20260618 22:26:56.137017 12859 raft_consensus.cc:2781] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:26:56.137176 12859 raft_consensus.cc:482] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:26:56.137290 12859 raft_consensus.cc:3037] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:56.139436 12859 raft_consensus.cc:504] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } }
I20260618 22:26:56.139998 12859 leader_election.cc:288] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [CANDIDATE]: Term 1 election: Requested vote from peers 3881c132165a41b0bfef176cccc7ad45 (127.10.242.193:40533), 41a89fa29b614e2ea824d5df3ad6b6fa (127.10.242.195:34241)
I20260618 22:26:56.140516 12773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a32c6efbb81946eea0252297bfbe93a5" candidate_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa"
I20260618 22:26:56.140559 12491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a32c6efbb81946eea0252297bfbe93a5" candidate_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3881c132165a41b0bfef176cccc7ad45"
I20260618 22:26:56.140758 12491 raft_consensus.cc:3037] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:56.140759 12773 raft_consensus.cc:3037] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:56.143360 12491 raft_consensus.cc:2454] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2f52598a2b44e38b97ba0305df5b0c in term 1.
I20260618 22:26:56.143572 12773 raft_consensus.cc:2454] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2f52598a2b44e38b97ba0305df5b0c in term 1.
I20260618 22:26:56.143770 12564 leader_election.cc:302] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3881c132165a41b0bfef176cccc7ad45, ba2f52598a2b44e38b97ba0305df5b0c; no voters:
I20260618 22:26:56.144049 12859 raft_consensus.cc:2781] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:56.144187 12859 raft_consensus.cc:686] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [term 1 LEADER]: Becoming Leader. State: Replica: ba2f52598a2b44e38b97ba0305df5b0c, State: Running, Role: LEADER
I20260618 22:26:56.144459 12859 consensus_queue.cc:237] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } }
I20260618 22:26:56.146217 12844 tablet_replica.cc:439] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c: TxnStatusTablet state changed. Reason: New leader ba2f52598a2b44e38b97ba0305df5b0c. Latest consensus state: current_term: 1 leader_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } }
I20260618 22:26:56.146533 12844 tablet_replica.cc:442] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:26:56.147548 12882 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260618 22:26:56.147792 12882 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260618 22:26:56.147746 12333 catalog_manager.cc:5495] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c reported cstate change: term changed from 0 to 1, leader changed from <none> to ba2f52598a2b44e38b97ba0305df5b0c (127.10.242.194). New cstate: current_term: 1 leader_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } health_report { overall_health: UNKNOWN } } }
I20260618 22:26:56.165916 12877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:56.160938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:45365 (local address 127.0.0.1:44902)
0618 22:26:56.161197 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:56.161216 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:56.161360 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:56.161715 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:56.161720 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:56.161731 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:26:56.161896 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:56.161904 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.162963 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.162969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:56.164019 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:56.164034 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.164785 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.164791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:56.164977 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:56.165033 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:26:56.165408 (+ 375us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:26:56.165427 (+ 19us) client_negotiation.cc:769] Sending connection context
0618 22:26:56.165498 (+ 71us) client_negotiation.cc:240] Negotiation successful
0618 22:26:56.165640 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260618 22:26:56.165916 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:56.160969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44902 (local address 127.10.242.194:45365)
0618 22:26:56.161215 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:56.161222 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:56.161288 (+ 66us) server_negotiation.cc:407] Connection header received
0618 22:26:56.161527 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:56.161533 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:56.161600 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:56.161672 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:56.162014 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.162796 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.164145 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.164643 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.164761 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:56.165222 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:56.165323 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:56.165404 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:56.165528 (+ 124us) server_negotiation.cc:299] Negotiation successful
0618 22:26:56.165612 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:26:56.170181 12773 raft_consensus.cc:1261] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 1 FOLLOWER]: Refusing update from remote peer ba2f52598a2b44e38b97ba0305df5b0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:56.170260 12491 raft_consensus.cc:1261] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45 [term 1 FOLLOWER]: Refusing update from remote peer ba2f52598a2b44e38b97ba0305df5b0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:56.170783 12844 consensus_queue.cc:1035] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:56.171456 12844 consensus_queue.cc:1035] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:56.174273 12845 tablet_replica.cc:439] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45: TxnStatusTablet state changed. Reason: New leader ba2f52598a2b44e38b97ba0305df5b0c. Latest consensus state: current_term: 1 leader_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } }
I20260618 22:26:56.174551 12845 tablet_replica.cc:442] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:26:56.176024 12859 tablet_replica.cc:439] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } }
I20260618 22:26:56.176335 12859 tablet_replica.cc:442] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:26:56.178236 12854 mvcc.cc:205] Tried to move back new op lower bound from 7298341339827048448 to 7298341339729932288. Current Snapshot: MvccSnapshot[applied={T|T < 7298341339827048448 or (T in {7298341339827048448})}]
I20260618 22:26:56.178545 12845 tablet_replica.cc:439] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } }
I20260618 22:26:56.178781 12845 tablet_replica.cc:442] T a32c6efbb81946eea0252297bfbe93a5 P 3881c132165a41b0bfef176cccc7ad45: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:26:56.185197 12846 tablet_replica.cc:439] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa: TxnStatusTablet state changed. Reason: New leader ba2f52598a2b44e38b97ba0305df5b0c. Latest consensus state: current_term: 1 leader_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } }
I20260618 22:26:56.185482 12846 tablet_replica.cc:442] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:26:56.186352 12849 tablet_replica.cc:439] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } }
I20260618 22:26:56.186717 12849 tablet_replica.cc:442] T a32c6efbb81946eea0252297bfbe93a5 P ba2f52598a2b44e38b97ba0305df5b0c: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:26:56.188709 12846 tablet_replica.cc:439] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3881c132165a41b0bfef176cccc7ad45" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 40533 } } peers { permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 } } peers { permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 } } }
I20260618 22:26:56.188937 12846 tablet_replica.cc:442] T a32c6efbb81946eea0252297bfbe93a5 P 41a89fa29b614e2ea824d5df3ad6b6fa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:26:56.197576 12835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:56.191356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47106 (local address 127.10.242.193:40533)
0618 22:26:56.191505 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:56.191509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:56.191525 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:26:56.191615 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:56.191618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:56.191699 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:56.191817 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:56.192243 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.192966 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.195366 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.195762 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.195864 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:56.196341 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:56.196887 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:56.196994 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:56.197203 (+ 209us) server_negotiation.cc:299] Negotiation successful
0618 22:26:56.197283 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:26:56.213481 12888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:56.207862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:45365 (local address 127.10.242.193:49027)
0618 22:26:56.208451 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:56.208470 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:56.208607 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:56.208916 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:56.208921 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:56.208932 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:26:56.209315 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:56.209326 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.210384 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.210388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:56.211463 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:56.211475 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.212232 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.212238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:56.212364 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:56.212425 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:26:56.212872 (+ 447us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:26:56.212883 (+ 11us) client_negotiation.cc:769] Sending connection context
0618 22:26:56.212972 (+ 89us) client_negotiation.cc:240] Negotiation successful
0618 22:26:56.213144 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":297,"threads_started":1}
I20260618 22:26:56.213495 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:56.207766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:49027 (local address 127.10.242.194:45365)
0618 22:26:56.207929 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:56.207933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:56.208563 (+ 630us) server_negotiation.cc:407] Connection header received
0618 22:26:56.208736 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:56.208741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:56.208816 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:56.208892 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:56.209463 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.210256 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.211611 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.212062 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.212189 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:56.212591 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:56.212723 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:56.212869 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:56.213005 (+ 136us) server_negotiation.cc:299] Negotiation successful
0618 22:26:56.213103 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260618 22:26:56.224402 12773 raft_consensus.cc:1261] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 1 FOLLOWER]: Refusing update from remote peer 3881c132165a41b0bfef176cccc7ad45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:56.224437 12635 raft_consensus.cc:1261] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c [term 1 FOLLOWER]: Refusing update from remote peer 3881c132165a41b0bfef176cccc7ad45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:56.225353 12845 consensus_queue.cc:1035] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41a89fa29b614e2ea824d5df3ad6b6fa" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 34241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:56.225683 12870 consensus_queue.cc:1035] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2f52598a2b44e38b97ba0305df5b0c" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 45365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:56.263865 12835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:56.256788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:54253 (local address 127.10.242.193:40533)
0618 22:26:56.256964 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:56.256970 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:56.257711 (+ 741us) server_negotiation.cc:407] Connection header received
0618 22:26:56.257916 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:56.257933 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:56.258000 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:56.258072 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:56.259198 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.260031 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.261706 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.262133 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.262237 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:56.263218 (+ 981us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:56.263354 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:56.263440 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:56.263594 (+ 154us) server_negotiation.cc:299] Negotiation successful
0618 22:26:56.263660 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260618 22:26:56.264228 12894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:56.256874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:40533 (local address 127.10.242.194:54253)
0618 22:26:56.257610 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:56.257626 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:56.257771 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:56.258143 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:56.258148 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:56.258160 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:26:56.259042 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:56.259055 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.260156 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.260162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:56.261576 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:56.261590 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.262789 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.262796 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:56.262979 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:56.263044 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:26:56.263468 (+ 424us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:26:56.263480 (+ 12us) client_negotiation.cc:769] Sending connection context
0618 22:26:56.263577 (+ 97us) client_negotiation.cc:240] Negotiation successful
0618 22:26:56.263740 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":418,"threads_started":1}
I20260618 22:26:56.311707 12334 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:26:56.311980 12334 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:26:56.315455 12334 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69805a4c96284ab6af68e887cdd744f7: Sending DeleteTablet for 3 replicas of tablet 220568d04c1f4f5796092c833adcb471
I20260618 22:26:56.316576 12471 tablet_service.cc:1514] Processing DeleteTablet for tablet 220568d04c1f4f5796092c833adcb471 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:56 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:55990
I20260618 22:26:56.316865 12863 tablet_replica.cc:330] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45: stopping tablet replica
I20260618 22:26:56.316987 12748 tablet_service.cc:1514] Processing DeleteTablet for tablet 220568d04c1f4f5796092c833adcb471 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:56 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:50014
I20260618 22:26:56.317063 12863 raft_consensus.cc:2229] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:26:56.317574 12863 raft_consensus.cc:2258] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:26:56.317858 12862 tablet_replica.cc:330] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa: stopping tablet replica
I20260618 22:26:56.318037 12862 raft_consensus.cc:2229] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:26:56.318205 12862 raft_consensus.cc:2258] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:26:56.319020 12610 tablet_service.cc:1514] Processing DeleteTablet for tablet 220568d04c1f4f5796092c833adcb471 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:56 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:49594
I20260618 22:26:56.319299 12864 tablet_replica.cc:330] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c: stopping tablet replica
I20260618 22:26:56.319478 12864 raft_consensus.cc:2229] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:26:56.319672 12864 raft_consensus.cc:2258] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:26:56.321255 12862 ts_tablet_manager.cc:1905] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:26:56.322441 12863 ts_tablet_manager.cc:1905] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:26:56.325888 12864 ts_tablet_manager.cc:1905] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:26:56.329187 12863 ts_tablet_manager.cc:1918] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:26:56.329345 12863 log.cc:1196] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/220568d04c1f4f5796092c833adcb471
I20260618 22:26:56.329768 12863 ts_tablet_manager.cc:1939] T 220568d04c1f4f5796092c833adcb471 P 3881c132165a41b0bfef176cccc7ad45: Deleting consensus metadata
I20260618 22:26:56.330920 12321 catalog_manager.cc:4841] TS 3881c132165a41b0bfef176cccc7ad45 (127.10.242.193:40533): tablet 220568d04c1f4f5796092c833adcb471 (table test-table [id=5cdd6c5eb16f4478a8eb9c8b8608aa89]) successfully deleted
I20260618 22:26:56.333586 12864 ts_tablet_manager.cc:1918] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:26:56.333746 12864 log.cc:1196] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/220568d04c1f4f5796092c833adcb471
I20260618 22:26:56.334153 12864 ts_tablet_manager.cc:1939] T 220568d04c1f4f5796092c833adcb471 P ba2f52598a2b44e38b97ba0305df5b0c: Deleting consensus metadata
I20260618 22:26:56.335227 12321 catalog_manager.cc:4841] TS ba2f52598a2b44e38b97ba0305df5b0c (127.10.242.194:45365): tablet 220568d04c1f4f5796092c833adcb471 (table test-table [id=5cdd6c5eb16f4478a8eb9c8b8608aa89]) successfully deleted
I20260618 22:26:56.336838 12862 ts_tablet_manager.cc:1918] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:26:56.337038 12862 log.cc:1196] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/220568d04c1f4f5796092c833adcb471
I20260618 22:26:56.337538 12862 ts_tablet_manager.cc:1939] T 220568d04c1f4f5796092c833adcb471 P 41a89fa29b614e2ea824d5df3ad6b6fa: Deleting consensus metadata
I20260618 22:26:56.338606 12321 catalog_manager.cc:4841] TS 41a89fa29b614e2ea824d5df3ad6b6fa (127.10.242.195:34241): tablet 220568d04c1f4f5796092c833adcb471 (table test-table [id=5cdd6c5eb16f4478a8eb9c8b8608aa89]) successfully deleted
Jun 18 22:26:56 dist-test-slave-0956 krb5kdc[12289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821615, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:26:56.352526 12835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:56.322525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47116 (local address 127.10.242.193:40533)
0618 22:26:56.325261 (+ 2736us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:56.325267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:56.331823 (+ 6556us) server_negotiation.cc:407] Connection header received
0618 22:26:56.332116 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:56.332122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:56.332200 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:56.332274 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:56.334403 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.335532 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.337389 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.338064 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.338157 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:56.346303 (+ 8146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:56.346316 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:56.346332 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:56.346374 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:56.349493 (+ 3119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:56.350235 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:56.350242 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:56.350245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:56.350408 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:56.350784 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:56.350789 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:56.350792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:56.351164 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:56.351577 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:56.351911 (+ 334us) server_negotiation.cc:299] Negotiation successful
0618 22:26:56.352121 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2583}
W20260618 22:26:56.353284 12536 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47116
I20260618 22:26:56.369060 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:56.356479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47388 (local address 127.10.242.254:46085)
0618 22:26:56.356649 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:56.356655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:56.356897 (+ 242us) server_negotiation.cc:407] Connection header received
0618 22:26:56.357044 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:56.357050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:56.357150 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:56.357245 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:56.359028 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.359885 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.361063 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.361523 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.361632 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:56.363904 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:56.363910 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:56.363913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:56.363951 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:56.366379 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:56.367031 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:56.367035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:56.367037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:56.367174 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:56.367560 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:56.367565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:56.367569 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:56.367901 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:56.368158 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:56.368489 (+ 331us) server_negotiation.cc:299] Negotiation successful
0618 22:26:56.368724 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260618 22:26:56.369628 12334 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47388
I20260618 22:26:56.371484 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 12405
I20260618 22:26:56.384858 12835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:56.373396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47122 (local address 127.10.242.193:40533)
0618 22:26:56.373552 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:56.373557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:56.373575 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:26:56.373631 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:56.373636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:56.373700 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:56.373792 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:56.374949 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.375655 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.376791 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.377261 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.377366 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:56.379453 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:56.379464 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:56.379469 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:56.379507 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:56.382130 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:56.382778 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:56.382784 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:56.382789 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:56.382924 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:56.383291 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:56.383296 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:56.383299 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:56.383669 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:56.383879 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:56.384282 (+ 403us) server_negotiation.cc:299] Negotiation successful
0618 22:26:56.384504 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260618 22:26:56.385430 12536 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47122
W20260618 22:26:56.385911 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 12405: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:26:56.386071 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 12405
I20260618 22:26:56.405576 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 12545
Jun 18 22:26:56 dist-test-slave-0956 krb5kdc[12289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821615, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.194@KRBTEST.COM
I20260618 22:26:56.420383 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:56.406375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44912 (local address 127.10.242.194:45365)
0618 22:26:56.406552 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:56.406558 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:56.406624 (+ 66us) server_negotiation.cc:407] Connection header received
0618 22:26:56.406864 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:56.406868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:56.406943 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:56.407026 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:56.408133 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.408924 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.410190 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.410620 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.410734 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:56.414527 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:56.414544 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:56.414558 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:56.414600 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:56.417823 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:56.418499 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:56.418506 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:56.418508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:56.418620 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:56.418971 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:56.418975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:56.418977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:56.419325 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:56.419593 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:56.419865 (+ 272us) server_negotiation.cc:299] Negotiation successful
0618 22:26:56.420066 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260618 22:26:56.420990 12675 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44912
W20260618 22:26:56.421608 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 12545: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:26:56.421787 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 12545
I20260618 22:26:56.438055 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 12684
Jun 18 22:26:56 dist-test-slave-0956 krb5kdc[12289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821615, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.195@KRBTEST.COM
I20260618 22:26:56.452230 12833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:56.438762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37414 (local address 127.10.242.195:34241)
0618 22:26:56.438909 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:56.438914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:56.438930 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:26:56.439056 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:56.439059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:56.439119 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:56.439200 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:56.440357 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.441156 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.442387 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:56.442814 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:56.442909 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:56.446992 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:56.447008 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:56.447023 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:56.447059 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:56.449914 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:56.450554 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:56.450562 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:56.450565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:56.450736 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:56.451035 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:56.451038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:56.451040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:56.451411 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:56.451683 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:56.451844 (+ 161us) server_negotiation.cc:299] Negotiation successful
0618 22:26:56.451995 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260618 22:26:56.452884 12814 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37414
W20260618 22:26:56.453594 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 12684: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:26:56.453725 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 12684
I20260618 22:26:56.466723 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 12302
I20260618 22:26:56.467784 12374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:26:56.578990 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 12302
2026-06-18T22:26:56Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4253 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:56 dist-test-slave-0956 krb5kdc[12906](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:56 dist-test-slave-0956 krb5kdc[12906](info): set up 2 sockets
Jun 18 22:26:56 dist-test-slave-0956 krb5kdc[12906](info): commencing operation
krb5kdc: starting...
W20260618 22:26:57.618343 11211 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.008s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260618 22:26:57.934880 11211 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.316s user 0.000s sys 0.004s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260618 22:26:58.252388 11211 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.317s user 0.000s sys 0.004s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260618 22:26:58.462021 11211 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.209s user 0.000s sys 0.004s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:26:58 dist-test-slave-0956 krb5kdc[12906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:26:58.560896 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:37899
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:46693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:37899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:58.880935 12919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:58.881284 12919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:58.881417 12919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:58.888657 12919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:58.888774 12919 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260618 22:26:58.888849 12919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:58.888931 12919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:58.888998 12919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:58.889097 12919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:58.899070 12919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:46693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:37899
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:37899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:26:58.900311 12919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:58.901701 12919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:58.912900 12927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:58.912983 12925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:58.912958 12924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:58.914023 12919 server_base.cc:1034] running on GCE node
I20260618 22:26:58.914839 12919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:58.916182 12919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:58.917394 12919 hybrid_clock.cc:648] HybridClock initialized: now 1781821618917328 us; error 44 us; skew 500 ppm
I20260618 22:26:58.917696 12919 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:26:58 dist-test-slave-0956 krb5kdc[12906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821618, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:58.920627 12919 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:26:58.922056 12919 webserver.cc:460] Webserver started at http://127.10.242.254:45703/ using document root <none> and password file <none>
I20260618 22:26:58.922720 12919 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:58.922813 12919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:58.923092 12919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:58.926296 12919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "079be3f013834f218fd9d5063fe112d9"
format_stamp: "Formatted at 2026-06-18 22:26:58 on dist-test-slave-0956"
server_key: "32d52e42e70782a9b1f471bac652ea20"
server_key_iv: "ad87f6227f6813f9f75d2c985c0613ac"
server_key_version: "encryptionkey@0"
I20260618 22:26:58.927289 12919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "079be3f013834f218fd9d5063fe112d9"
format_stamp: "Formatted at 2026-06-18 22:26:58 on dist-test-slave-0956"
server_key: "32d52e42e70782a9b1f471bac652ea20"
server_key_iv: "ad87f6227f6813f9f75d2c985c0613ac"
server_key_version: "encryptionkey@0"
I20260618 22:26:58.932643 12919 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:26:58.936204 12934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:58.937806 12919 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:26:58.938019 12919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "079be3f013834f218fd9d5063fe112d9"
format_stamp: "Formatted at 2026-06-18 22:26:58 on dist-test-slave-0956"
server_key: "32d52e42e70782a9b1f471bac652ea20"
server_key_iv: "ad87f6227f6813f9f75d2c985c0613ac"
server_key_version: "encryptionkey@0"
I20260618 22:26:58.938216 12919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:58.956357 12919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:58.960738 12919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:58.961047 12919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:58.982730 12919 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:37899
I20260618 22:26:58.982753 12995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:37899 every 8 connection(s)
I20260618 22:26:58.984157 12919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:58.988005 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 12919
I20260618 22:26:58.988111 12996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:58.988217 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:26:58.988981 11211 external_mini_cluster.cc:1428] Setting key 18ff0468cd2da8839bde5b90ec78c00a
I20260618 22:26:59.001951 12996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9: Bootstrap starting.
Jun 18 22:26:58 dist-test-slave-0956 krb5kdc[12906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:26:59.005957 12996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:59.007058 12996 log.cc:824] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:59.010461 12996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9: No bootstrap required, opened a new log
I20260618 22:26:59.013320 12999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:58.991141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43932 (local address 127.10.242.254:37899)
0618 22:26:58.991921 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:58.991933 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:58.991980 (+ 47us) server_negotiation.cc:407] Connection header received
0618 22:26:58.994108 (+ 2128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:58.994132 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:58.994567 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:58.994920 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:58.996598 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:58.998022 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:58.999790 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:59.000498 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:59.000633 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:59.005375 (+ 4742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:59.005385 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:59.005402 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:59.005448 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:59.008884 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:59.009833 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:59.009839 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:59.009846 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:59.010019 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:59.010442 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:59.010445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:59.010451 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:59.011102 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:59.011323 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:59.011811 (+ 488us) server_negotiation.cc:299] Negotiation successful
0618 22:26:59.012200 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":373,"threads_started":1}
I20260618 22:26:59.018321 12996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "079be3f013834f218fd9d5063fe112d9" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 37899 } }
I20260618 22:26:59.018700 12996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:59.018800 12996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 079be3f013834f218fd9d5063fe112d9, State: Initialized, Role: FOLLOWER
I20260618 22:26:59.019392 12996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "079be3f013834f218fd9d5063fe112d9" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 37899 } }
I20260618 22:26:59.019609 12996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:59.019739 12996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:59.019891 12996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:59.022092 12996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "079be3f013834f218fd9d5063fe112d9" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 37899 } }
I20260618 22:26:59.022603 12996 leader_election.cc:302] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 079be3f013834f218fd9d5063fe112d9; no voters:
I20260618 22:26:59.023149 12996 leader_election.cc:288] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:59.023296 13001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:59.023932 13001 raft_consensus.cc:686] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9 [term 1 LEADER]: Becoming Leader. State: Replica: 079be3f013834f218fd9d5063fe112d9, State: Running, Role: LEADER
I20260618 22:26:59.024425 13001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "079be3f013834f218fd9d5063fe112d9" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 37899 } }
I20260618 22:26:59.025148 12996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:59.027347 13002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "079be3f013834f218fd9d5063fe112d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "079be3f013834f218fd9d5063fe112d9" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 37899 } } }
I20260618 22:26:59.027809 13002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:59.027417 13003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 079be3f013834f218fd9d5063fe112d9. Latest consensus state: current_term: 1 leader_uuid: "079be3f013834f218fd9d5063fe112d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "079be3f013834f218fd9d5063fe112d9" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 37899 } } }
I20260618 22:26:59.027981 13003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:59.029021 13008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:59.034175 13008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:59.043731 13008 catalog_manager.cc:1347] Generated new cluster ID: 1bcd2e77329c4c9ab3fdcc42dd5b941a
I20260618 22:26:59.043862 13008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:59.054199 13008 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:59.055586 13008 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:59.074927 13008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9: Generated new TSK 0
I20260618 22:26:59.075858 13008 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
I20260618 22:26:59.128765 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:37899
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:46693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:59.457345 13022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:59.457636 13022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:59.457737 13022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:59.464979 13022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:59.465129 13022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:59.465238 13022 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260618 22:26:59.465293 13022 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:26:59.465353 13022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:26:59.475211 13022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:46693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.242.254:37899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:26:59.476468 13022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:59.477736 13022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:59.490435 13027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:59.490408 13030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:59.490464 13028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:59.490922 13022 server_base.cc:1034] running on GCE node
I20260618 22:26:59.491568 13022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:59.492498 13022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:59.493733 13022 hybrid_clock.cc:648] HybridClock initialized: now 1781821619493661 us; error 59 us; skew 500 ppm
I20260618 22:26:59.494100 13022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:26:59 dist-test-slave-0956 krb5kdc[12906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821619, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:59.497331 13022 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:26:59.499037 13022 webserver.cc:460] Webserver started at http://127.10.242.193:33637/ using document root <none> and password file <none>
I20260618 22:26:59.499754 13022 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:59.499882 13022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:59.500190 13022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:59.503338 13022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "9f0f878d5d97442cae587020545849ed"
format_stamp: "Formatted at 2026-06-18 22:26:59 on dist-test-slave-0956"
server_key: "ac41114f1fb4d07b9362a1a3bd015f14"
server_key_iv: "7c4ec815b7108f179f46964ea2bfeee9"
server_key_version: "encryptionkey@0"
I20260618 22:26:59.504141 13022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "9f0f878d5d97442cae587020545849ed"
format_stamp: "Formatted at 2026-06-18 22:26:59 on dist-test-slave-0956"
server_key: "ac41114f1fb4d07b9362a1a3bd015f14"
server_key_iv: "7c4ec815b7108f179f46964ea2bfeee9"
server_key_version: "encryptionkey@0"
I20260618 22:26:59.509867 13022 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260618 22:26:59.513562 13037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:59.515084 13022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:26:59.515254 13022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "9f0f878d5d97442cae587020545849ed"
format_stamp: "Formatted at 2026-06-18 22:26:59 on dist-test-slave-0956"
server_key: "ac41114f1fb4d07b9362a1a3bd015f14"
server_key_iv: "7c4ec815b7108f179f46964ea2bfeee9"
server_key_version: "encryptionkey@0"
I20260618 22:26:59.515458 13022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:59.534017 13022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:59.538712 13022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:59.538998 13022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:59.541832 13022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:59.541962 13022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:59.542124 13022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:59.542215 13022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:26:59 dist-test-slave-0956 krb5kdc[12906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821619, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:26:59.566294 13046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:59.542447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:50663 (local address 127.10.242.254:37899)
0618 22:26:59.543163 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:59.543169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:59.544353 (+ 1184us) server_negotiation.cc:407] Connection header received
0618 22:26:59.546440 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:59.546445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:59.546536 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:59.546634 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:59.549149 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:59.550006 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:59.553096 (+ 3090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:59.553640 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:59.553790 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:59.558874 (+ 5084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:59.558883 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:59.558888 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:59.558939 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:59.562198 (+ 3259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:59.563369 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:59.563374 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:59.563378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:59.563533 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:59.564141 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:59.564146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:59.564150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:59.564524 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:59.564661 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:59.565711 (+ 1050us) server_negotiation.cc:299] Negotiation successful
0618 22:26:59.565938 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":486,"threads_started":1}
I20260618 22:26:59.567452 13047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:59.543242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:37899 (local address 127.10.242.193:50663)
0618 22:26:59.544169 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:59.544219 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:59.546125 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:59.546964 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:59.546978 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:59.547754 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:59.548853 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:59.548883 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:59.550228 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:59.550235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:59.552854 (+ 2619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:59.552878 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:59.553930 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:59.553942 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:59.554293 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:59.555253 (+ 960us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:59.555291 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:59.558552 (+ 3261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:59.562397 (+ 3845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:59.562408 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:59.562432 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:59.563217 (+ 785us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:59.563666 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:59.563675 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:59.563679 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:59.564014 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:59.564745 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:59.564754 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:59.565333 (+ 579us) client_negotiation.cc:769] Sending connection context
0618 22:26:59.565623 (+ 290us) client_negotiation.cc:240] Negotiation successful
0618 22:26:59.566209 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":374,"threads_started":1}
I20260618 22:26:59.593403 13022 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:42231
I20260618 22:26:59.593462 13157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:42231 every 8 connection(s)
I20260618 22:26:59.594961 13022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:26:59.597229 13158 heartbeater.cc:347] Connected to a master server at 127.10.242.254:37899
I20260618 22:26:59.597474 13158 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:59.598310 13158 heartbeater.cc:511] Master 127.10.242.254:37899 requested a full tablet report, sending...
I20260618 22:26:59.598357 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 13022
I20260618 22:26:59.598551 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:26:59.599222 11211 external_mini_cluster.cc:1428] Setting key 866b3b65359efa51b9488b89972b753e
I20260618 22:26:59.599262 13134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:59.581377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:37899 (local address 127.10.242.193:55795)
0618 22:26:59.585882 (+ 4505us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:59.585909 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:59.586067 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:59.586466 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:59.586471 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:59.587007 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:59.587284 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:59.587294 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:59.588678 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:59.588684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:59.590394 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:59.590411 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:59.591422 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:59.591429 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:59.591565 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:59.592181 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:59.592206 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:59.593373 (+ 1167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:59.596442 (+ 3069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:59.596447 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:59.596451 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:59.596908 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:59.597335 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:59.597340 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:59.597344 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:59.597472 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:59.598177 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:59.598182 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:59.598367 (+ 185us) client_negotiation.cc:769] Sending connection context
0618 22:26:59.598500 (+ 133us) client_negotiation.cc:240] Negotiation successful
0618 22:26:59.598815 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4355,"thread_start_us":4228,"threads_started":1}
I20260618 22:26:59.600142 13046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:59.581342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:55795 (local address 127.10.242.254:37899)
0618 22:26:59.581546 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:59.581553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:59.586034 (+ 4481us) server_negotiation.cc:407] Connection header received
0618 22:26:59.586174 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:59.586181 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:59.586282 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:59.586410 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:59.587439 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:59.588544 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:59.590592 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:59.591196 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:59.591343 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:59.593613 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:59.593626 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:59.593631 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:59.593673 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:59.596312 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:59.597023 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:59.597028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:59.597030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:59.597228 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:59.597589 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:59.597593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:59.597597 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:59.597948 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:59.598113 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:59.599636 (+ 1523us) server_negotiation.cc:299] Negotiation successful
0618 22:26:59.599804 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260618 22:26:59.600668 12951 ts_manager.cc:194] Registered new tserver with Master: 9f0f878d5d97442cae587020545849ed (127.10.242.193:42231)
I20260618 22:26:59.602196 12951 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:50663
WARNING: no policy specified for kudu/127.10.242.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
I20260618 22:26:59.636539 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:37899
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:46693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:59.961684 13162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:59.962013 13162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:59.962145 13162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:59.969444 13162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:59.969559 13162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:59.969681 13162 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260618 22:26:59.969736 13162 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:26:59.969815 13162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:26:59.979583 13162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:46693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.242.254:37899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:26:59.980969 13162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:59.982406 13162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:59.996402 13167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:59.996518 13168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:59.996757 13170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:59.997313 13162 server_base.cc:1034] running on GCE node
I20260618 22:26:59.997932 13162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:59.999094 13162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:00.000435 13162 hybrid_clock.cc:648] HybridClock initialized: now 1781821620000300 us; error 127 us; skew 500 ppm
I20260618 22:27:00.000895 13162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:26:59 dist-test-slave-0956 krb5kdc[12906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821619, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:00.004964 13162 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:27:00.006732 13162 webserver.cc:460] Webserver started at http://127.10.242.194:34483/ using document root <none> and password file <none>
I20260618 22:27:00.007481 13162 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:00.007615 13162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:00.007934 13162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:00.011138 13162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "fe6f8874615b456bbb65c66e009135f6"
format_stamp: "Formatted at 2026-06-18 22:27:00 on dist-test-slave-0956"
server_key: "b1910d0b66d2fc2e82c9810ed2dc198d"
server_key_iv: "612cb2eae62ecd32801a0fcfc517b6d3"
server_key_version: "encryptionkey@0"
I20260618 22:27:00.011972 13162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "fe6f8874615b456bbb65c66e009135f6"
format_stamp: "Formatted at 2026-06-18 22:27:00 on dist-test-slave-0956"
server_key: "b1910d0b66d2fc2e82c9810ed2dc198d"
server_key_iv: "612cb2eae62ecd32801a0fcfc517b6d3"
server_key_version: "encryptionkey@0"
I20260618 22:27:00.018146 13162 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260618 22:27:00.022121 13177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:00.023763 13162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:27:00.023953 13162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "fe6f8874615b456bbb65c66e009135f6"
format_stamp: "Formatted at 2026-06-18 22:27:00 on dist-test-slave-0956"
server_key: "b1910d0b66d2fc2e82c9810ed2dc198d"
server_key_iv: "612cb2eae62ecd32801a0fcfc517b6d3"
server_key_version: "encryptionkey@0"
I20260618 22:27:00.024166 13162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:00.054780 13162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:00.059121 13162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:00.059381 13162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:00.061933 13162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:00.062045 13162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:00.062161 13162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:00.062258 13162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:27:00 dist-test-slave-0956 krb5kdc[12906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821619, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:00.085886 13046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.062646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:41855 (local address 127.10.242.254:37899)
0618 22:27:00.062837 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:00.062843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:00.064324 (+ 1481us) server_negotiation.cc:407] Connection header received
0618 22:27:00.066819 (+ 2495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:00.066827 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:00.066941 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:00.067055 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:00.069487 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.070576 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.073004 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.073581 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.073709 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.078779 (+ 5070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:00.078785 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:00.078788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:00.078823 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:00.082432 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:00.083314 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:00.083321 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:00.083326 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:00.083515 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:00.083965 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:00.083970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:00.083973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:00.084377 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:00.084543 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:00.085307 (+ 764us) server_negotiation.cc:299] Negotiation successful
0618 22:27:00.085530 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:27:00.087270 13186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.063253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:37899 (local address 127.10.242.194:41855)
0618 22:27:00.064125 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:00.064174 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:00.066454 (+ 2280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:00.067378 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:00.067393 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:00.068230 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:00.069222 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.069253 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.070783 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.070790 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.072853 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.072866 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.074043 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.074049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.074321 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.075114 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:00.075151 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:00.078485 (+ 3334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:00.082637 (+ 4152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:00.082649 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:00.082671 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:00.083166 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:00.083615 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:00.083618 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:00.083621 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:00.083841 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:00.084504 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:00.084511 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:00.084897 (+ 386us) client_negotiation.cc:769] Sending connection context
0618 22:27:00.085188 (+ 291us) client_negotiation.cc:240] Negotiation successful
0618 22:27:00.085792 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":358,"threads_started":1}
I20260618 22:27:00.101370 13162 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:42087
I20260618 22:27:00.101372 13296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:42087 every 8 connection(s)
I20260618 22:27:00.102782 13162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:27:00.104560 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 13162
I20260618 22:27:00.104753 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:27:00.105275 11211 external_mini_cluster.cc:1428] Setting key 9bbb27214cf8d604a8e3ab24f8f633a7
I20260618 22:27:00.104915 13297 heartbeater.cc:347] Connected to a master server at 127.10.242.254:37899
I20260618 22:27:00.106194 13297 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:00.107012 13297 heartbeater.cc:511] Master 127.10.242.254:37899 requested a full tablet report, sending...
I20260618 22:27:00.108528 12951 ts_manager.cc:194] Registered new tserver with Master: fe6f8874615b456bbb65c66e009135f6 (127.10.242.194:42087)
I20260618 22:27:00.109491 13046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.094230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:40039 (local address 127.10.242.254:37899)
0618 22:27:00.094869 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:00.094876 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:00.095435 (+ 559us) server_negotiation.cc:407] Connection header received
0618 22:27:00.095644 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:00.095650 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:00.095738 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:00.095840 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:00.097481 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.098608 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.100674 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.101342 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.101470 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.103298 (+ 1828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:00.103305 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:00.103309 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:00.103349 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:00.106652 (+ 3303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:00.107300 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:00.107305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:00.107309 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:00.107486 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:00.107877 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:00.107882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:00.107886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:00.108320 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:00.108476 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:00.108917 (+ 441us) server_negotiation.cc:299] Negotiation successful
0618 22:27:00.109176 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488}
I20260618 22:27:00.109567 13276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.094477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:37899 (local address 127.10.242.194:40039)
0618 22:27:00.095317 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:00.095334 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:00.095526 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:00.096065 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:00.096070 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:00.096750 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:00.097247 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.097269 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.098825 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.098834 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.100469 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.100488 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.101569 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.101575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.101737 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.102307 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:00.102326 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:00.103086 (+ 760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:00.106760 (+ 3674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:00.106766 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:00.106771 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:00.107197 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:00.107610 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:00.107618 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:00.107621 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:00.107767 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:00.108460 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:00.108465 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:00.108652 (+ 187us) client_negotiation.cc:769] Sending connection context
0618 22:27:00.108800 (+ 148us) client_negotiation.cc:240] Negotiation successful
0618 22:27:00.109156 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":516,"threads_started":1}
I20260618 22:27:00.109572 12951 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:41855
WARNING: no policy specified for kudu/127.10.242.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
I20260618 22:27:00.144001 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:37899
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:46693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:00.458913 13301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:00.459194 13301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:00.459297 13301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:00.466499 13301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:00.466617 13301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:00.466701 13301 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260618 22:27:00.466750 13301 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:27:00.466822 13301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:27:00.476632 13301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:46693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.242.254:37899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:00.477931 13301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:00.479282 13301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:00.492213 13309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:00.492250 13306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:00.492169 13307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:00.493587 13301 server_base.cc:1034] running on GCE node
I20260618 22:27:00.494215 13301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:00.495177 13301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:00.496377 13301 hybrid_clock.cc:648] HybridClock initialized: now 1781821620496324 us; error 47 us; skew 500 ppm
I20260618 22:27:00.496687 13301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:00 dist-test-slave-0956 krb5kdc[12906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821620, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:00.500104 13301 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:27:00.501688 13301 webserver.cc:460] Webserver started at http://127.10.242.195:35027/ using document root <none> and password file <none>
I20260618 22:27:00.502416 13301 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:00.502537 13301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:00.502846 13301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:00.505831 13301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "9bb994c822f14df7a83e33284f452f51"
format_stamp: "Formatted at 2026-06-18 22:27:00 on dist-test-slave-0956"
server_key: "daf857eb1b53fa460b8cd015d0aeb15e"
server_key_iv: "976240225d1f410e90544ddbe913591a"
server_key_version: "encryptionkey@0"
I20260618 22:27:00.506582 13301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "9bb994c822f14df7a83e33284f452f51"
format_stamp: "Formatted at 2026-06-18 22:27:00 on dist-test-slave-0956"
server_key: "daf857eb1b53fa460b8cd015d0aeb15e"
server_key_iv: "976240225d1f410e90544ddbe913591a"
server_key_version: "encryptionkey@0"
I20260618 22:27:00.511787 13301 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:27:00.515344 13316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:00.516716 13301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:27:00.516902 13301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "9bb994c822f14df7a83e33284f452f51"
format_stamp: "Formatted at 2026-06-18 22:27:00 on dist-test-slave-0956"
server_key: "daf857eb1b53fa460b8cd015d0aeb15e"
server_key_iv: "976240225d1f410e90544ddbe913591a"
server_key_version: "encryptionkey@0"
I20260618 22:27:00.517131 13301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:00.532544 13301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:00.536777 13301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:00.537048 13301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:00.539724 13301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:00.539839 13301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:00.539963 13301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:00.540050 13301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:27:00 dist-test-slave-0956 krb5kdc[12906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821620, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:00.566138 13325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.541270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:37899 (local address 127.10.242.195:51709)
0618 22:27:00.542185 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:00.542223 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:00.544381 (+ 2158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:00.545457 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:00.545474 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:00.546640 (+ 1166us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:00.547689 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.547721 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.549193 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.549200 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.551727 (+ 2527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.551748 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.552911 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.552918 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.553210 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.554058 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:00.554091 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:00.557195 (+ 3104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:00.560789 (+ 3594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:00.560804 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:00.560828 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:00.561623 (+ 795us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:00.562052 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:00.562058 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:00.562062 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:00.562358 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:00.563027 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:00.563036 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:00.563541 (+ 505us) client_negotiation.cc:769] Sending connection context
0618 22:27:00.563918 (+ 377us) client_negotiation.cc:240] Negotiation successful
0618 22:27:00.564479 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":395,"threads_started":1}
I20260618 22:27:00.566974 13046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.540572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:51709 (local address 127.10.242.254:37899)
0618 22:27:00.540755 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:00.540760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:00.542332 (+ 1572us) server_negotiation.cc:407] Connection header received
0618 22:27:00.544737 (+ 2405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:00.544744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:00.544837 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:00.544956 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:00.547976 (+ 3020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.548951 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.551923 (+ 2972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.552456 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.552589 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.557469 (+ 4880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:00.557474 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:00.557478 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:00.557517 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:00.560620 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:00.561788 (+ 1168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:00.561794 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:00.561798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:00.561937 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:00.562503 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:00.562509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:00.562512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:00.562898 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:00.563033 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:00.566553 (+ 3520us) server_negotiation.cc:299] Negotiation successful
0618 22:27:00.566734 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260618 22:27:00.575850 13301 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:40331
I20260618 22:27:00.575850 13435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:40331 every 8 connection(s)
I20260618 22:27:00.577641 13301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:27:00.580047 13436 heartbeater.cc:347] Connected to a master server at 127.10.242.254:37899
I20260618 22:27:00.580268 13436 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:00.580894 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 13301
I20260618 22:27:00.581068 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:27:00.581142 13436 heartbeater.cc:511] Master 127.10.242.254:37899 requested a full tablet report, sending...
I20260618 22:27:00.581555 11211 external_mini_cluster.cc:1428] Setting key f0d27dc13179d06c21a6fa3ffa849b74
I20260618 22:27:00.582824 12951 ts_manager.cc:194] Registered new tserver with Master: 9bb994c822f14df7a83e33284f452f51 (127.10.242.195:40331)
I20260618 22:27:00.583901 12951 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:51709
I20260618 22:27:00.584767 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:27:00.587383 13431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.573737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:37899 (local address 127.10.242.195:49753)
0618 22:27:00.574342 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:00.574362 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:00.574497 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:00.575025 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:00.575029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:00.575569 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:00.575929 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.575942 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.577349 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.577354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.578673 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.578691 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.579525 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.579531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.579698 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.580344 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:00.580366 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:00.581385 (+ 1019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:00.584984 (+ 3599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:00.584990 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:00.584994 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:00.585540 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:00.585938 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:00.585942 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:00.585945 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:00.586094 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:00.586603 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:00.586608 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:00.586755 (+ 147us) client_negotiation.cc:769] Sending connection context
0618 22:27:00.586876 (+ 121us) client_negotiation.cc:240] Negotiation successful
0618 22:27:00.587090 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":315,"threads_started":1}
I20260618 22:27:00.587527 13046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.574187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:49753 (local address 127.10.242.254:37899)
0618 22:27:00.574384 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:00.574390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:00.574417 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:27:00.574637 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:00.574643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:00.574737 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:00.574827 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:00.576127 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.577167 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.578888 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.579370 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.579482 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.581567 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:00.581573 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:00.581578 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:00.581620 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:00.584855 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:00.585660 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:00.585666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:00.585670 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:00.585829 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:00.586189 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:00.586193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:00.586197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:00.586486 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:00.586624 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:00.586994 (+ 370us) server_negotiation.cc:299] Negotiation successful
0618 22:27:00.587192 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260618 22:27:00.605849 13158 heartbeater.cc:503] Master 127.10.242.254:37899 was elected leader, sending a full tablet report...
Jun 18 22:27:00 dist-test-slave-0956 krb5kdc[12906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 18 22:27:00 dist-test-slave-0956 krb5kdc[12906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:00.631174 13046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.616307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43940 (local address 127.10.242.254:37899)
0618 22:27:00.616533 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:00.616539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:00.616663 (+ 124us) server_negotiation.cc:407] Connection header received
0618 22:27:00.616898 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:00.616903 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:00.616970 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:00.617062 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:00.618503 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.619306 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.620638 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.621221 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.621338 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.625708 (+ 4370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:00.625714 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:00.625718 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:00.625762 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:00.628505 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:00.629315 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:00.629319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:00.629321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:00.629457 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:00.629874 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:00.629877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:00.629879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:00.630161 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:00.630337 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:00.630735 (+ 398us) server_negotiation.cc:299] Negotiation successful
0618 22:27:00.630909 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260618 22:27:00.635591 12950 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:27:00.642520 12950 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:27:00.676232 13452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.655278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35792 (local address 127.10.242.194:42087)
0618 22:27:00.666317 (+ 11039us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:00.666340 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:00.667038 (+ 698us) server_negotiation.cc:407] Connection header received
0618 22:27:00.667143 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:00.667151 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:00.667449 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:00.667654 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:00.668602 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.670232 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.673781 (+ 3549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.675086 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.675260 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.675444 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:00.675772 (+ 328us) server_negotiation.cc:299] Negotiation successful
0618 22:27:00.675936 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10883,"thread_start_us":10759,"threads_started":1}
I20260618 22:27:00.676383 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.654974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:42087 (local address 127.0.0.1:35792)
0618 22:27:00.666775 (+ 11801us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:00.666792 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:00.666911 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:00.667898 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:00.667906 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:00.667935 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:00.668324 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.668349 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.670477 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.670483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.673600 (+ 3117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.673614 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.675307 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.675315 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.675514 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.675569 (+ 55us) client_negotiation.cc:769] Sending connection context
0618 22:27:00.675710 (+ 141us) client_negotiation.cc:240] Negotiation successful
0618 22:27:00.675919 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11657,"mutex_wait_us":558,"thread_start_us":10267,"threads_started":1}
I20260618 22:27:00.677168 13451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.654584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:40331 (local address 127.0.0.1:55708)
0618 22:27:00.664718 (+ 10134us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:00.664786 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:00.664989 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:00.668728 (+ 3739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:00.668732 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:00.668754 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:00.669174 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.669186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.671662 (+ 2476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.671668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.673917 (+ 2249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.673929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.675598 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.675604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.675773 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.675805 (+ 32us) client_negotiation.cc:769] Sending connection context
0618 22:27:00.675885 (+ 80us) client_negotiation.cc:240] Negotiation successful
0618 22:27:00.676020 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9801,"mutex_wait_us":5,"thread_start_us":11905,"threads_started":1}
I20260618 22:27:00.680019 13450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.654840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55708 (local address 127.10.242.195:40331)
0618 22:27:00.667890 (+ 13050us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:00.667898 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:00.667928 (+ 30us) server_negotiation.cc:407] Connection header received
0618 22:27:00.668017 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:00.668024 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:00.668253 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:00.668449 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:00.669423 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.671243 (+ 1820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.674099 (+ 2856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.675139 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.675388 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.675571 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:00.676206 (+ 635us) server_negotiation.cc:299] Negotiation successful
0618 22:27:00.676374 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12876,"thread_start_us":12699,"threads_started":1}
I20260618 22:27:00.680563 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.667244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42548 (local address 127.10.242.193:42231)
0618 22:27:00.671237 (+ 3993us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:00.671245 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:00.673171 (+ 1926us) server_negotiation.cc:407] Connection header received
0618 22:27:00.673265 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:00.673271 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:00.673510 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:00.673698 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:00.674355 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.675765 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.678223 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.679599 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.679778 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.679937 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:00.680113 (+ 176us) server_negotiation.cc:299] Negotiation successful
0618 22:27:00.680276 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3844,"thread_start_us":3738,"threads_started":1}
I20260618 22:27:00.680878 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.667932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:42231 (local address 127.0.0.1:42548)
0618 22:27:00.672694 (+ 4762us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:00.672714 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:00.672842 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:00.673697 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:00.673701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:00.673726 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:00.674152 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.674165 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.675926 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.675930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.678028 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.678043 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.679799 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.679804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.679927 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.679953 (+ 26us) client_negotiation.cc:769] Sending connection context
0618 22:27:00.680045 (+ 92us) client_negotiation.cc:240] Negotiation successful
0618 22:27:00.680359 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4631,"mutex_wait_us":1,"thread_start_us":4470,"threads_started":1}
I20260618 22:27:00.680647 13227 tablet_service.cc:1467] Processing CreateTablet for tablet b5c89a919784421397e9f4c8945d4a31 (DEFAULT_TABLE table=test-table [id=507a10ce0b9e49e1bf16ea4c396f2482]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:00.682192 13227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5c89a919784421397e9f4c8945d4a31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:00.683861 13088 tablet_service.cc:1467] Processing CreateTablet for tablet b5c89a919784421397e9f4c8945d4a31 (DEFAULT_TABLE table=test-table [id=507a10ce0b9e49e1bf16ea4c396f2482]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:00.683879 13366 tablet_service.cc:1467] Processing CreateTablet for tablet b5c89a919784421397e9f4c8945d4a31 (DEFAULT_TABLE table=test-table [id=507a10ce0b9e49e1bf16ea4c396f2482]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:00.685151 13088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5c89a919784421397e9f4c8945d4a31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:00.685374 13366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5c89a919784421397e9f4c8945d4a31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:00.696336 13455 tablet_bootstrap.cc:492] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51: Bootstrap starting.
I20260618 22:27:00.698320 13456 tablet_bootstrap.cc:492] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6: Bootstrap starting.
I20260618 22:27:00.700265 13455 tablet_bootstrap.cc:654] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:00.701052 13457 tablet_bootstrap.cc:492] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed: Bootstrap starting.
I20260618 22:27:00.701565 13456 tablet_bootstrap.cc:654] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:00.702035 13455 log.cc:824] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:00.702771 13456 log.cc:824] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:00.704370 13457 tablet_bootstrap.cc:654] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:00.704767 13455 tablet_bootstrap.cc:492] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51: No bootstrap required, opened a new log
I20260618 22:27:00.705118 13455 ts_tablet_manager.cc:1397] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260618 22:27:00.705468 13457 log.cc:824] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:00.707064 13456 tablet_bootstrap.cc:492] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6: No bootstrap required, opened a new log
I20260618 22:27:00.707319 13456 ts_tablet_manager.cc:1397] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260618 22:27:00.708948 13457 tablet_bootstrap.cc:492] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed: No bootstrap required, opened a new log
I20260618 22:27:00.709216 13457 ts_tablet_manager.cc:1397] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260618 22:27:00.712667 13455 raft_consensus.cc:348] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } }
I20260618 22:27:00.713045 13455 raft_consensus.cc:374] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:00.713182 13455 raft_consensus.cc:729] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bb994c822f14df7a83e33284f452f51, State: Initialized, Role: FOLLOWER
I20260618 22:27:00.713757 13455 consensus_queue.cc:260] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } }
I20260618 22:27:00.714790 13436 heartbeater.cc:503] Master 127.10.242.254:37899 was elected leader, sending a full tablet report...
I20260618 22:27:00.715512 13455 ts_tablet_manager.cc:1428] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51: Time spent starting tablet: real 0.010s user 0.000s sys 0.009s
I20260618 22:27:00.717361 13456 raft_consensus.cc:348] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } }
I20260618 22:27:00.717800 13456 raft_consensus.cc:374] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:00.717900 13456 raft_consensus.cc:729] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe6f8874615b456bbb65c66e009135f6, State: Initialized, Role: FOLLOWER
I20260618 22:27:00.718518 13456 consensus_queue.cc:260] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } }
I20260618 22:27:00.719763 13297 heartbeater.cc:503] Master 127.10.242.254:37899 was elected leader, sending a full tablet report...
I20260618 22:27:00.719662 13457 raft_consensus.cc:348] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } }
I20260618 22:27:00.720170 13457 raft_consensus.cc:374] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:00.720263 13457 raft_consensus.cc:729] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0f878d5d97442cae587020545849ed, State: Initialized, Role: FOLLOWER
I20260618 22:27:00.720376 13456 ts_tablet_manager.cc:1428] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260618 22:27:00.721130 13457 consensus_queue.cc:260] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } }
I20260618 22:27:00.723392 13457 ts_tablet_manager.cc:1428] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
I20260618 22:27:00.806761 13461 raft_consensus.cc:482] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:27:00.807085 13461 raft_consensus.cc:504] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } }
I20260618 22:27:00.808784 13461 leader_election.cc:288] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f0f878d5d97442cae587020545849ed (127.10.242.193:42231), fe6f8874615b456bbb65c66e009135f6 (127.10.242.194:42087)
I20260618 22:27:00.815001 13325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.809006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:42231 (local address 127.10.242.195:58397)
0618 22:27:00.809313 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:00.809332 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:00.809464 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:00.809799 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:00.809803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:00.809831 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:00.810045 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.810055 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.811283 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.811289 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.812834 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.812847 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.814235 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.814241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.814454 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.814491 (+ 37us) client_negotiation.cc:769] Sending connection context
0618 22:27:00.814589 (+ 98us) client_negotiation.cc:240] Negotiation successful
0618 22:27:00.814722 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174}
I20260618 22:27:00.815001 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.809247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:58397 (local address 127.10.242.193:42231)
0618 22:27:00.809459 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:00.809465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:00.809490 (+ 25us) server_negotiation.cc:407] Connection header received
0618 22:27:00.809583 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:00.809589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:00.809663 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:00.809759 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:00.810168 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.811126 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.812986 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.814074 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.814206 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.814264 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:00.814614 (+ 350us) server_negotiation.cc:299] Negotiation successful
0618 22:27:00.814706 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260618 22:27:00.815591 13452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.809312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:57743 (local address 127.10.242.194:42087)
0618 22:27:00.809486 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:00.809491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:00.810045 (+ 554us) server_negotiation.cc:407] Connection header received
0618 22:27:00.810193 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:00.810198 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:00.810271 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:00.810360 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:00.810797 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.811809 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.813719 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.814712 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.814853 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.814931 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:00.815264 (+ 333us) server_negotiation.cc:299] Negotiation successful
0618 22:27:00.815420 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:27:00.816052 13464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.809216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:42087 (local address 127.10.242.195:57743)
0618 22:27:00.809937 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:00.809954 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:00.810073 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:00.810370 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:00.810373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:00.810394 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:00.810651 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.810658 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.811950 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.811954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.813580 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.813594 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.814923 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.814929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.815089 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.815120 (+ 31us) client_negotiation.cc:769] Sending connection context
0618 22:27:00.815740 (+ 620us) client_negotiation.cc:240] Negotiation successful
0618 22:27:00.815840 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":400,"threads_started":1}
I20260618 22:27:00.815985 13108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5c89a919784421397e9f4c8945d4a31" candidate_uuid: "9bb994c822f14df7a83e33284f452f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0f878d5d97442cae587020545849ed" is_pre_election: true
I20260618 22:27:00.816485 13108 raft_consensus.cc:2454] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bb994c822f14df7a83e33284f452f51 in term 0.
I20260618 22:27:00.816918 13247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5c89a919784421397e9f4c8945d4a31" candidate_uuid: "9bb994c822f14df7a83e33284f452f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6f8874615b456bbb65c66e009135f6" is_pre_election: true
I20260618 22:27:00.817129 13318 leader_election.cc:302] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bb994c822f14df7a83e33284f452f51, 9f0f878d5d97442cae587020545849ed; no voters:
I20260618 22:27:00.817406 13247 raft_consensus.cc:2454] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bb994c822f14df7a83e33284f452f51 in term 0.
I20260618 22:27:00.817516 13461 raft_consensus.cc:2781] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:27:00.817654 13461 raft_consensus.cc:482] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:27:00.817752 13461 raft_consensus.cc:3037] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:00.820124 13461 raft_consensus.cc:504] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } }
I20260618 22:27:00.820788 13461 leader_election.cc:288] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [CANDIDATE]: Term 1 election: Requested vote from peers 9f0f878d5d97442cae587020545849ed (127.10.242.193:42231), fe6f8874615b456bbb65c66e009135f6 (127.10.242.194:42087)
I20260618 22:27:00.821337 13108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5c89a919784421397e9f4c8945d4a31" candidate_uuid: "9bb994c822f14df7a83e33284f452f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0f878d5d97442cae587020545849ed"
I20260618 22:27:00.821420 13247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5c89a919784421397e9f4c8945d4a31" candidate_uuid: "9bb994c822f14df7a83e33284f452f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6f8874615b456bbb65c66e009135f6"
I20260618 22:27:00.821552 13108 raft_consensus.cc:3037] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:00.821614 13247 raft_consensus.cc:3037] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:00.823752 13247 raft_consensus.cc:2454] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bb994c822f14df7a83e33284f452f51 in term 1.
I20260618 22:27:00.823916 13108 raft_consensus.cc:2454] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bb994c822f14df7a83e33284f452f51 in term 1.
I20260618 22:27:00.824167 13317 leader_election.cc:302] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bb994c822f14df7a83e33284f452f51, fe6f8874615b456bbb65c66e009135f6; no voters:
I20260618 22:27:00.824515 13461 raft_consensus.cc:2781] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:00.824960 13461 raft_consensus.cc:686] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [term 1 LEADER]: Becoming Leader. State: Replica: 9bb994c822f14df7a83e33284f452f51, State: Running, Role: LEADER
I20260618 22:27:00.825575 13461 consensus_queue.cc:237] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } }
W20260618 22:27:00.829344 13437 tablet.cc:2369] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:27:00.830427 12951 catalog_manager.cc:5495] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9bb994c822f14df7a83e33284f452f51 (127.10.242.195). New cstate: current_term: 1 leader_uuid: "9bb994c822f14df7a83e33284f452f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } health_report { overall_health: UNKNOWN } } }
W20260618 22:27:00.847162 13159 tablet.cc:2369] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:27:00.854933 13298 tablet.cc:2369] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 18 22:27:00 dist-test-slave-0956 krb5kdc[12906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821618, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:00.900646 13046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.887624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43952 (local address 127.10.242.254:37899)
0618 22:27:00.887834 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:00.887839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:00.888338 (+ 499us) server_negotiation.cc:407] Connection header received
0618 22:27:00.888530 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:00.888534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:00.888601 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:00.888681 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:00.889870 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.890610 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.891837 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.892249 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.892356 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.895567 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:00.895571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:00.895574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:00.895603 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:00.898069 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:00.898739 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:00.898744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:00.898746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:00.898873 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:00.899192 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:00.899196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:00.899199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:00.899507 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:00.899633 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:00.900036 (+ 403us) server_negotiation.cc:299] Negotiation successful
0618 22:27:00.900346 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":122,"server-negotiator.queue_time_us":59}
I20260618 22:27:00.900683 13472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.887650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:37899 (local address 127.0.0.1:43952)
0618 22:27:00.888229 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:00.888244 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:00.888414 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:00.888729 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:00.888734 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:00.889496 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:00.889733 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.889743 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.890727 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.890733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.891699 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.891710 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.892423 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.892428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.892568 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.893210 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:00.893226 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:00.895401 (+ 2175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:00.898208 (+ 2807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:00.898223 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:00.898231 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:00.898627 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:00.898971 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:00.898975 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:00.898979 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:00.899098 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:00.899626 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:00.899636 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:00.899798 (+ 162us) client_negotiation.cc:769] Sending connection context
0618 22:27:00.899939 (+ 141us) client_negotiation.cc:240] Negotiation successful
0618 22:27:00.900212 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":312,"threads_started":1}
I20260618 22:27:00.906038 12951 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:43952:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260618 22:27:00.907011 12951 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:27:00.913367 13088 tablet_service.cc:1467] Processing CreateTablet for tablet 958d093edf3d4c9183dea1244b865d5c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=136187c8765e4d8a83d9dda8c04d0c69]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:27:00.913620 13366 tablet_service.cc:1467] Processing CreateTablet for tablet 958d093edf3d4c9183dea1244b865d5c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=136187c8765e4d8a83d9dda8c04d0c69]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:27:00.913424 13227 tablet_service.cc:1467] Processing CreateTablet for tablet 958d093edf3d4c9183dea1244b865d5c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=136187c8765e4d8a83d9dda8c04d0c69]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:27:00.914353 13227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 958d093edf3d4c9183dea1244b865d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:00.914356 13366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 958d093edf3d4c9183dea1244b865d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:00.914358 13088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 958d093edf3d4c9183dea1244b865d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:00.919909 13456 tablet_bootstrap.cc:492] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6: Bootstrap starting.
I20260618 22:27:00.921262 13457 tablet_bootstrap.cc:492] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed: Bootstrap starting.
I20260618 22:27:00.921514 13456 tablet_bootstrap.cc:654] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:00.922827 13457 tablet_bootstrap.cc:654] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:00.924013 13455 tablet_bootstrap.cc:492] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51: Bootstrap starting.
I20260618 22:27:00.925788 13455 tablet_bootstrap.cc:654] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:00.928121 13455 tablet_bootstrap.cc:492] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51: No bootstrap required, opened a new log
I20260618 22:27:00.928148 13457 tablet_bootstrap.cc:492] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed: No bootstrap required, opened a new log
I20260618 22:27:00.928272 13457 ts_tablet_manager.cc:1397] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.002s
I20260618 22:27:00.928272 13455 ts_tablet_manager.cc:1397] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260618 22:27:00.929035 13457 raft_consensus.cc:348] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:00.929032 13455 raft_consensus.cc:348] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:00.929297 13457 raft_consensus.cc:374] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:00.929297 13455 raft_consensus.cc:374] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:00.929399 13455 raft_consensus.cc:729] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bb994c822f14df7a83e33284f452f51, State: Initialized, Role: FOLLOWER
I20260618 22:27:00.929390 13457 raft_consensus.cc:729] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0f878d5d97442cae587020545849ed, State: Initialized, Role: FOLLOWER
I20260618 22:27:00.929618 13455 consensus_queue.cc:260] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:00.929618 13457 consensus_queue.cc:260] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:00.930158 13457 ts_tablet_manager.cc:1428] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:27:00.930198 13455 ts_tablet_manager.cc:1428] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 22:27:00.930061 13461 tablet_replica.cc:439] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } }
I20260618 22:27:00.930369 13461 tablet_replica.cc:442] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:27:00.930691 13463 tablet_replica.cc:439] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } }
I20260618 22:27:00.931042 13463 tablet_replica.cc:442] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:27:00.931722 13456 tablet_bootstrap.cc:492] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6: No bootstrap required, opened a new log
I20260618 22:27:00.931875 13456 ts_tablet_manager.cc:1397] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.005s
I20260618 22:27:00.932823 13456 raft_consensus.cc:348] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:00.933065 13456 raft_consensus.cc:374] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:00.933189 13456 raft_consensus.cc:729] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe6f8874615b456bbb65c66e009135f6, State: Initialized, Role: FOLLOWER
I20260618 22:27:00.933480 13456 consensus_queue.cc:260] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:00.933952 13462 tablet_replica.cc:439] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } }
I20260618 22:27:00.934298 13462 tablet_replica.cc:442] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:27:00.935667 13456 ts_tablet_manager.cc:1428] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260618 22:27:00.978184 13463 raft_consensus.cc:482] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:27:00.978438 13463 raft_consensus.cc:504] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:00.979986 13463 leader_election.cc:288] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe6f8874615b456bbb65c66e009135f6 (127.10.242.194:42087), 9bb994c822f14df7a83e33284f452f51 (127.10.242.195:40331)
I20260618 22:27:00.987498 13450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.980423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:38091 (local address 127.10.242.195:40331)
0618 22:27:00.980644 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:00.980650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:00.981209 (+ 559us) server_negotiation.cc:407] Connection header received
0618 22:27:00.981439 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:00.981444 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:00.981504 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:00.981597 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:00.982223 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.983330 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.985850 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.986624 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.986776 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.986849 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:00.987130 (+ 281us) server_negotiation.cc:299] Negotiation successful
0618 22:27:00.987251 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260618 22:27:00.987618 13476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.980268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:40331 (local address 127.10.242.193:38091)
0618 22:27:00.981070 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:00.981121 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:00.981288 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:00.981623 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:00.981628 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:00.981648 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:00.982029 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.982044 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.983474 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.983479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.985651 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.985665 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.986830 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.986836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.986987 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.987011 (+ 24us) client_negotiation.cc:769] Sending connection context
0618 22:27:00.987090 (+ 79us) client_negotiation.cc:240] Negotiation successful
0618 22:27:00.987220 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":35,"thread_start_us":540,"threads_started":1}
I20260618 22:27:00.988221 13452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.980786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:54661 (local address 127.10.242.194:42087)
0618 22:27:00.980956 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:00.980961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:00.981534 (+ 573us) server_negotiation.cc:407] Connection header received
0618 22:27:00.981634 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:00.981639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:00.981702 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:00.981780 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:00.982360 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.983174 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.985819 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.987060 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.987206 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.987273 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:00.987801 (+ 528us) server_negotiation.cc:299] Negotiation successful
0618 22:27:00.987938 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260618 22:27:00.988936 13477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:00.980357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:42087 (local address 127.10.242.193:54661)
0618 22:27:00.981445 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:00.981458 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:00.981542 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:00.981808 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:00.981812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:00.981839 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:00.982204 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.982214 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.983311 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.983316 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.985650 (+ 2334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:00.985665 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:00.987427 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:00.987434 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:00.987611 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:00.987644 (+ 33us) client_negotiation.cc:769] Sending connection context
0618 22:27:00.988397 (+ 753us) client_negotiation.cc:240] Negotiation successful
0618 22:27:00.988553 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":958,"mutex_wait_us":33,"thread_start_us":768,"threads_started":1}
I20260618 22:27:00.988806 13386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "958d093edf3d4c9183dea1244b865d5c" candidate_uuid: "9f0f878d5d97442cae587020545849ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bb994c822f14df7a83e33284f452f51" is_pre_election: true
I20260618 22:27:00.989204 13386 raft_consensus.cc:2454] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f0f878d5d97442cae587020545849ed in term 0.
I20260618 22:27:00.989487 13247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "958d093edf3d4c9183dea1244b865d5c" candidate_uuid: "9f0f878d5d97442cae587020545849ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6f8874615b456bbb65c66e009135f6" is_pre_election: true
I20260618 22:27:00.989698 13247 raft_consensus.cc:2454] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f0f878d5d97442cae587020545849ed in term 0.
I20260618 22:27:00.989777 13039 leader_election.cc:302] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bb994c822f14df7a83e33284f452f51, 9f0f878d5d97442cae587020545849ed; no voters:
I20260618 22:27:00.990195 13463 raft_consensus.cc:2781] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:27:00.990320 13463 raft_consensus.cc:482] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:27:00.990406 13463 raft_consensus.cc:3037] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:00.993443 13463 raft_consensus.cc:504] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:00.994107 13463 leader_election.cc:288] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [CANDIDATE]: Term 1 election: Requested vote from peers fe6f8874615b456bbb65c66e009135f6 (127.10.242.194:42087), 9bb994c822f14df7a83e33284f452f51 (127.10.242.195:40331)
I20260618 22:27:00.994568 13386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "958d093edf3d4c9183dea1244b865d5c" candidate_uuid: "9f0f878d5d97442cae587020545849ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bb994c822f14df7a83e33284f452f51"
I20260618 22:27:00.994590 13247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "958d093edf3d4c9183dea1244b865d5c" candidate_uuid: "9f0f878d5d97442cae587020545849ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6f8874615b456bbb65c66e009135f6"
I20260618 22:27:00.994743 13386 raft_consensus.cc:3037] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:00.994798 13247 raft_consensus.cc:3037] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:00.997334 13386 raft_consensus.cc:2454] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f0f878d5d97442cae587020545849ed in term 1.
I20260618 22:27:00.997648 13247 raft_consensus.cc:2454] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f0f878d5d97442cae587020545849ed in term 1.
I20260618 22:27:00.997701 13039 leader_election.cc:302] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bb994c822f14df7a83e33284f452f51, 9f0f878d5d97442cae587020545849ed; no voters:
I20260618 22:27:00.997995 13463 raft_consensus.cc:2781] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:00.998456 13463 raft_consensus.cc:686] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [term 1 LEADER]: Becoming Leader. State: Replica: 9f0f878d5d97442cae587020545849ed, State: Running, Role: LEADER
I20260618 22:27:00.998955 13463 consensus_queue.cc:237] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:01.002210 13478 tablet_replica.cc:439] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed: TxnStatusTablet state changed. Reason: New leader 9f0f878d5d97442cae587020545849ed. Latest consensus state: current_term: 1 leader_uuid: "9f0f878d5d97442cae587020545849ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } }
I20260618 22:27:01.002676 13478 tablet_replica.cc:442] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:27:01.003453 13480 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260618 22:27:01.003667 13480 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260618 22:27:01.003878 12951 catalog_manager.cc:5495] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f0f878d5d97442cae587020545849ed (127.10.242.193). New cstate: current_term: 1 leader_uuid: "9f0f878d5d97442cae587020545849ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } health_report { overall_health: UNKNOWN } } }
I20260618 22:27:01.031551 13472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:01.026095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:42231 (local address 127.0.0.1:42554)
0618 22:27:01.026330 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:01.026349 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:01.026492 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:01.026967 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:01.026972 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:01.026983 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:27:01.027132 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:01.027139 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.028195 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.028200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:01.029269 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:01.029281 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.029989 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.029995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:01.030145 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:01.030201 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:27:01.031028 (+ 827us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:27:01.031049 (+ 21us) client_negotiation.cc:769] Sending connection context
0618 22:27:01.031127 (+ 78us) client_negotiation.cc:240] Negotiation successful
0618 22:27:01.031272 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260618 22:27:01.031587 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:01.026203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42554 (local address 127.10.242.193:42231)
0618 22:27:01.026376 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:01.026382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:01.026401 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:27:01.026687 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:01.026691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:01.026779 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:01.026895 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:27:01.027252 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.028077 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.029410 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.029842 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.029956 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:01.030388 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:27:01.030926 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:27:01.031051 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:01.031151 (+ 100us) server_negotiation.cc:299] Negotiation successful
0618 22:27:01.031234 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:27:01.039311 13247 raft_consensus.cc:1261] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Refusing update from remote peer 9f0f878d5d97442cae587020545849ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:01.039366 13386 raft_consensus.cc:1261] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51 [term 1 FOLLOWER]: Refusing update from remote peer 9f0f878d5d97442cae587020545849ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:01.040242 13478 consensus_queue.cc:1035] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:01.040674 13463 consensus_queue.cc:1035] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:01.050535 13462 tablet_replica.cc:439] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6: TxnStatusTablet state changed. Reason: New leader 9f0f878d5d97442cae587020545849ed. Latest consensus state: current_term: 1 leader_uuid: "9f0f878d5d97442cae587020545849ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } }
I20260618 22:27:01.050861 13462 tablet_replica.cc:442] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:27:01.053047 13461 tablet_replica.cc:439] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51: TxnStatusTablet state changed. Reason: New leader 9f0f878d5d97442cae587020545849ed. Latest consensus state: current_term: 1 leader_uuid: "9f0f878d5d97442cae587020545849ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } }
I20260618 22:27:01.053402 13461 tablet_replica.cc:442] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:27:01.062356 13484 mvcc.cc:205] Tried to move back new op lower bound from 7298341359767285760 to 7298341359615209472. Current Snapshot: MvccSnapshot[applied={T|T < 7298341359767285760}]
I20260618 22:27:01.086148 13450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:01.077778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.10.242.195:40331)
0618 22:27:01.077971 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:01.077977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:01.078669 (+ 692us) server_negotiation.cc:407] Connection header received
0618 22:27:01.078795 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:01.078800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:01.078918 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:01.079097 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:27:01.079687 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.080893 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.082970 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.083640 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.083777 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:01.084400 (+ 623us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:27:01.085151 (+ 751us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:27:01.085289 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:01.085726 (+ 437us) server_negotiation.cc:299] Negotiation successful
0618 22:27:01.085824 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260618 22:27:01.089329 13461 tablet_replica.cc:439] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9f0f878d5d97442cae587020545849ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } }
I20260618 22:27:01.089618 13461 tablet_replica.cc:442] T 958d093edf3d4c9183dea1244b865d5c P 9bb994c822f14df7a83e33284f452f51: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:27:01.094599 13489 tablet_replica.cc:439] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9f0f878d5d97442cae587020545849ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } }
I20260618 22:27:01.095219 13489 tablet_replica.cc:442] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:27:01.095870 13463 tablet_replica.cc:439] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9f0f878d5d97442cae587020545849ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } }
I20260618 22:27:01.096220 13463 tablet_replica.cc:442] T 958d093edf3d4c9183dea1244b865d5c P 9f0f878d5d97442cae587020545849ed: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:27:01.097746 13485 mvcc.cc:205] Tried to move back new op lower bound from 7298341359767285760 to 7298341359615209472. Current Snapshot: MvccSnapshot[applied={T|T < 7298341359767285760}]
I20260618 22:27:01.113914 13462 tablet_replica.cc:439] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9f0f878d5d97442cae587020545849ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } } }
I20260618 22:27:01.114187 13462 tablet_replica.cc:442] T 958d093edf3d4c9183dea1244b865d5c P fe6f8874615b456bbb65c66e009135f6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:27:01.127786 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:01.120049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:44503 (local address 127.10.242.193:42231)
0618 22:27:01.120244 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:01.120252 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:01.120813 (+ 561us) server_negotiation.cc:407] Connection header received
0618 22:27:01.121149 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:01.121155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:01.121247 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:01.121308 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:27:01.122025 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.123169 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.125372 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.126139 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.126246 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:01.127005 (+ 759us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:27:01.127150 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:27:01.127210 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:01.127425 (+ 215us) server_negotiation.cc:299] Negotiation successful
0618 22:27:01.127500 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:27:01.128029 13496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:01.120007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:42231 (local address 127.10.242.195:44503)
0618 22:27:01.120710 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:01.120731 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:01.120944 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:01.121580 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:01.121583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:01.121595 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:27:01.121836 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:01.121847 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.123352 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.123357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:01.125185 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:01.125203 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.126478 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.126485 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:01.126669 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:01.126754 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:27:01.127285 (+ 531us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:27:01.127296 (+ 11us) client_negotiation.cc:769] Sending connection context
0618 22:27:01.127375 (+ 79us) client_negotiation.cc:240] Negotiation successful
0618 22:27:01.127534 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":349,"threads_started":1}
I20260618 22:27:01.139243 13247 raft_consensus.cc:1261] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Refusing update from remote peer 9bb994c822f14df7a83e33284f452f51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:01.139585 13108 raft_consensus.cc:1261] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed [term 1 FOLLOWER]: Refusing update from remote peer 9bb994c822f14df7a83e33284f452f51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:01.139988 13465 consensus_queue.cc:1035] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:01.140441 13461 consensus_queue.cc:1035] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:01.177278 13450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:01.170587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:37253 (local address 127.10.242.195:40331)
0618 22:27:01.170769 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:01.170775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:01.171201 (+ 426us) server_negotiation.cc:407] Connection header received
0618 22:27:01.171353 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:01.171358 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:01.171440 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:01.171531 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:27:01.172516 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.173635 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.175185 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.175868 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.175995 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:01.176482 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:27:01.176623 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:27:01.176710 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:01.176862 (+ 152us) server_negotiation.cc:299] Negotiation successful
0618 22:27:01.176944 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260618 22:27:01.177886 13500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:01.170520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:40331 (local address 127.10.242.193:37253)
0618 22:27:01.171112 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:01.171128 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:01.171244 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:01.171575 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:01.171580 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:01.171593 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:27:01.171959 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:01.171972 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.173761 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.173765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:01.175047 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:01.175059 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.176058 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.176063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:01.176181 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:01.176265 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:27:01.176730 (+ 465us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:27:01.176741 (+ 11us) client_negotiation.cc:769] Sending connection context
0618 22:27:01.176836 (+ 95us) client_negotiation.cc:240] Negotiation successful
0618 22:27:01.177291 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":282,"threads_started":1}
I20260618 22:27:01.221303 12950 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:01.221719 12950 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:01.226111 12950 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9: Sending DeleteTablet for 3 replicas of tablet b5c89a919784421397e9f4c8945d4a31
I20260618 22:27:01.227485 13366 tablet_service.cc:1514] Processing DeleteTablet for tablet b5c89a919784421397e9f4c8945d4a31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:01 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:55708
I20260618 22:27:01.227587 13227 tablet_service.cc:1514] Processing DeleteTablet for tablet b5c89a919784421397e9f4c8945d4a31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:01 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:35792
I20260618 22:27:01.228626 13088 tablet_service.cc:1514] Processing DeleteTablet for tablet b5c89a919784421397e9f4c8945d4a31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:01 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:42548
I20260618 22:27:01.233799 12950 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:27:01.234885 12950 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:27:01.240962 13504 tablet_replica.cc:330] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6: stopping tablet replica
I20260618 22:27:01.241791 13504 raft_consensus.cc:2229] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:01.242395 13504 raft_consensus.cc:2258] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:01.244699 13504 ts_tablet_manager.cc:1905] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:01.245433 13503 tablet_replica.cc:330] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51: stopping tablet replica
I20260618 22:27:01.245975 13503 raft_consensus.cc:2229] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:27:01.246871 13227 tablet_service.cc:1467] Processing CreateTablet for tablet e51d9c50e09940e6bf4cd7cc4360fe05 (DEFAULT_TABLE table=test-table [id=75b30de20ab94926a3f44b436e700eff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:01.247412 13227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51d9c50e09940e6bf4cd7cc4360fe05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:01.247393 13087 tablet_service.cc:1467] Processing CreateTablet for tablet e51d9c50e09940e6bf4cd7cc4360fe05 (DEFAULT_TABLE table=test-table [id=75b30de20ab94926a3f44b436e700eff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:01.247876 13087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51d9c50e09940e6bf4cd7cc4360fe05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:01.253983 13365 tablet_service.cc:1467] Processing CreateTablet for tablet e51d9c50e09940e6bf4cd7cc4360fe05 (DEFAULT_TABLE table=test-table [id=75b30de20ab94926a3f44b436e700eff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:01.254684 13365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51d9c50e09940e6bf4cd7cc4360fe05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:01.258255 13457 tablet_bootstrap.cc:492] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed: Bootstrap starting.
I20260618 22:27:01.260157 13457 tablet_bootstrap.cc:654] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:01.261706 13455 tablet_bootstrap.cc:492] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51: Bootstrap starting.
I20260618 22:27:01.263612 13455 tablet_bootstrap.cc:654] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:01.264014 13505 tablet_replica.cc:330] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed: stopping tablet replica
I20260618 22:27:01.265604 13505 raft_consensus.cc:2229] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:01.265991 13505 raft_consensus.cc:2258] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:01.268836 13503 raft_consensus.cc:2258] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:01.268985 13505 ts_tablet_manager.cc:1905] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:01.273880 13456 tablet_bootstrap.cc:492] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6: Bootstrap starting.
I20260618 22:27:01.275570 13456 tablet_bootstrap.cc:654] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:01.279628 13505 ts_tablet_manager.cc:1918] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:27:01.279803 13505 log.cc:1196] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/b5c89a919784421397e9f4c8945d4a31
I20260618 22:27:01.280251 13505 ts_tablet_manager.cc:1939] T b5c89a919784421397e9f4c8945d4a31 P 9f0f878d5d97442cae587020545849ed: Deleting consensus metadata
I20260618 22:27:01.281956 12936 catalog_manager.cc:4841] TS 9f0f878d5d97442cae587020545849ed (127.10.242.193:42231): tablet b5c89a919784421397e9f4c8945d4a31 (table test-table [id=507a10ce0b9e49e1bf16ea4c396f2482]) successfully deleted
I20260618 22:27:01.286501 13503 ts_tablet_manager.cc:1905] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:01.290611 13457 tablet_bootstrap.cc:492] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed: No bootstrap required, opened a new log
I20260618 22:27:01.290819 13457 ts_tablet_manager.cc:1397] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed: Time spent bootstrapping tablet: real 0.033s user 0.009s sys 0.006s
I20260618 22:27:01.291589 13457 raft_consensus.cc:348] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:01.291741 13504 ts_tablet_manager.cc:1918] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:27:01.291816 13457 raft_consensus.cc:374] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:01.291886 13457 raft_consensus.cc:729] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0f878d5d97442cae587020545849ed, State: Initialized, Role: FOLLOWER
I20260618 22:27:01.291904 13504 log.cc:1196] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/b5c89a919784421397e9f4c8945d4a31
I20260618 22:27:01.292102 13457 consensus_queue.cc:260] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:01.292372 13504 ts_tablet_manager.cc:1939] T b5c89a919784421397e9f4c8945d4a31 P fe6f8874615b456bbb65c66e009135f6: Deleting consensus metadata
I20260618 22:27:01.292603 13457 ts_tablet_manager.cc:1428] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260618 22:27:01.293704 13456 tablet_bootstrap.cc:492] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6: No bootstrap required, opened a new log
I20260618 22:27:01.293923 13456 ts_tablet_manager.cc:1397] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.005s
I20260618 22:27:01.294699 13456 raft_consensus.cc:348] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:01.294993 13456 raft_consensus.cc:374] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:01.295555 12935 catalog_manager.cc:4841] TS fe6f8874615b456bbb65c66e009135f6 (127.10.242.194:42087): tablet b5c89a919784421397e9f4c8945d4a31 (table test-table [id=507a10ce0b9e49e1bf16ea4c396f2482]) successfully deleted
I20260618 22:27:01.296111 13455 tablet_bootstrap.cc:492] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51: No bootstrap required, opened a new log
I20260618 22:27:01.296336 13455 ts_tablet_manager.cc:1397] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.009s
I20260618 22:27:01.297214 13455 raft_consensus.cc:348] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:01.297523 13455 raft_consensus.cc:374] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:01.297680 13455 raft_consensus.cc:729] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bb994c822f14df7a83e33284f452f51, State: Initialized, Role: FOLLOWER
I20260618 22:27:01.298019 13455 consensus_queue.cc:260] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:01.298791 13455 ts_tablet_manager.cc:1428] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:27:01.301331 13503 ts_tablet_manager.cc:1918] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:27:01.301504 13503 log.cc:1196] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/b5c89a919784421397e9f4c8945d4a31
I20260618 22:27:01.302067 13503 ts_tablet_manager.cc:1939] T b5c89a919784421397e9f4c8945d4a31 P 9bb994c822f14df7a83e33284f452f51: Deleting consensus metadata
I20260618 22:27:01.302380 13456 raft_consensus.cc:729] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe6f8874615b456bbb65c66e009135f6, State: Initialized, Role: FOLLOWER
I20260618 22:27:01.302675 13456 consensus_queue.cc:260] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:01.303290 13456 ts_tablet_manager.cc:1428] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20260618 22:27:01.303925 12936 catalog_manager.cc:4841] TS 9bb994c822f14df7a83e33284f452f51 (127.10.242.195:40331): tablet b5c89a919784421397e9f4c8945d4a31 (table test-table [id=507a10ce0b9e49e1bf16ea4c396f2482]) successfully deleted
I20260618 22:27:01.429850 13462 raft_consensus.cc:482] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:27:01.430125 13462 raft_consensus.cc:504] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:01.431912 13462 leader_election.cc:288] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f0f878d5d97442cae587020545849ed (127.10.242.193:42231), 9bb994c822f14df7a83e33284f452f51 (127.10.242.195:40331)
I20260618 22:27:01.436848 13463 raft_consensus.cc:482] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:27:01.437036 13463 raft_consensus.cc:504] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:01.437870 13463 leader_election.cc:288] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe6f8874615b456bbb65c66e009135f6 (127.10.242.194:42087), 9bb994c822f14df7a83e33284f452f51 (127.10.242.195:40331)
I20260618 22:27:01.438659 13386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51d9c50e09940e6bf4cd7cc4360fe05" candidate_uuid: "9f0f878d5d97442cae587020545849ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bb994c822f14df7a83e33284f452f51" is_pre_election: true
I20260618 22:27:01.438921 13386 raft_consensus.cc:2454] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f0f878d5d97442cae587020545849ed in term 0.
I20260618 22:27:01.439631 13039 leader_election.cc:302] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bb994c822f14df7a83e33284f452f51, 9f0f878d5d97442cae587020545849ed; no voters:
I20260618 22:27:01.439997 13463 raft_consensus.cc:2781] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:27:01.440109 13463 raft_consensus.cc:482] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:27:01.440207 13463 raft_consensus.cc:3037] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:01.443887 13463 raft_consensus.cc:504] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:01.445569 13386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51d9c50e09940e6bf4cd7cc4360fe05" candidate_uuid: "9f0f878d5d97442cae587020545849ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bb994c822f14df7a83e33284f452f51"
I20260618 22:27:01.445591 13246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51d9c50e09940e6bf4cd7cc4360fe05" candidate_uuid: "9f0f878d5d97442cae587020545849ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6f8874615b456bbb65c66e009135f6" is_pre_election: true
I20260618 22:27:01.445820 13386 raft_consensus.cc:3037] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:01.445921 13246 raft_consensus.cc:2454] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f0f878d5d97442cae587020545849ed in term 0.
I20260618 22:27:01.446509 13246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51d9c50e09940e6bf4cd7cc4360fe05" candidate_uuid: "9f0f878d5d97442cae587020545849ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6f8874615b456bbb65c66e009135f6"
I20260618 22:27:01.446681 13246 raft_consensus.cc:3037] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:01.448971 13463 leader_election.cc:288] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [CANDIDATE]: Term 1 election: Requested vote from peers fe6f8874615b456bbb65c66e009135f6 (127.10.242.194:42087), 9bb994c822f14df7a83e33284f452f51 (127.10.242.195:40331)
I20260618 22:27:01.449059 13386 raft_consensus.cc:2454] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f0f878d5d97442cae587020545849ed in term 1.
I20260618 22:27:01.449602 13039 leader_election.cc:302] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bb994c822f14df7a83e33284f452f51, 9f0f878d5d97442cae587020545849ed; no voters:
I20260618 22:27:01.450011 13463 raft_consensus.cc:2781] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:01.450465 13463 raft_consensus.cc:686] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [term 1 LEADER]: Becoming Leader. State: Replica: 9f0f878d5d97442cae587020545849ed, State: Running, Role: LEADER
I20260618 22:27:01.450789 13463 consensus_queue.cc:237] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:01.451840 13246 raft_consensus.cc:2454] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f0f878d5d97442cae587020545849ed in term 1.
I20260618 22:27:01.460822 13450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:01.433573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:53979 (local address 127.10.242.195:40331)
0618 22:27:01.433767 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:01.433774 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:01.444959 (+ 11185us) server_negotiation.cc:407] Connection header received
0618 22:27:01.447717 (+ 2758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:01.447723 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:01.447792 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:01.447850 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:01.454312 (+ 6462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.455478 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.458299 (+ 2821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.459533 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.459669 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:01.459756 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:01.460361 (+ 605us) server_negotiation.cc:299] Negotiation successful
0618 22:27:01.460512 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260618 22:27:01.461441 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:01.432320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:43829 (local address 127.10.242.193:42231)
0618 22:27:01.432530 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:01.432536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:01.433072 (+ 536us) server_negotiation.cc:407] Connection header received
0618 22:27:01.433386 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:01.433392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:01.433467 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:01.433528 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:01.434326 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.435491 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.458293 (+ 22802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.460150 (+ 1857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.460372 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:01.460445 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:01.460973 (+ 528us) server_negotiation.cc:299] Negotiation successful
0618 22:27:01.461145 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260618 22:27:01.462195 13510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:01.433172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:40331 (local address 127.10.242.194:53979)
0618 22:27:01.444810 (+ 11638us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:01.444834 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:01.447517 (+ 2683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:01.448110 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:01.448114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:01.448141 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:01.454102 (+ 5961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:01.454120 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.455968 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.455974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:01.457971 (+ 1997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:01.457989 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.459967 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.459974 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:01.460159 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:01.460193 (+ 34us) client_negotiation.cc:769] Sending connection context
0618 22:27:01.461311 (+ 1118us) client_negotiation.cc:240] Negotiation successful
0618 22:27:01.461468 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11457,"mutex_wait_us":5592,"thread_start_us":11808,"threads_started":1}
I20260618 22:27:01.463297 13386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51d9c50e09940e6bf4cd7cc4360fe05" candidate_uuid: "fe6f8874615b456bbb65c66e009135f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bb994c822f14df7a83e33284f452f51" is_pre_election: true
I20260618 22:27:01.463646 13386 raft_consensus.cc:2379] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fe6f8874615b456bbb65c66e009135f6 in current term 1: Already voted for candidate 9f0f878d5d97442cae587020545849ed in this term.
I20260618 22:27:01.462633 13509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:01.432167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:42231 (local address 127.10.242.194:43829)
0618 22:27:01.432956 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:01.432976 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:01.433212 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:01.433718 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:01.433723 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:01.433749 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:01.434112 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:01.434124 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.435692 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.435697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:01.453855 (+ 18158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:01.458138 (+ 4283us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.460586 (+ 2448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.460593 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:01.460775 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:01.460819 (+ 44us) client_negotiation.cc:769] Sending connection context
0618 22:27:01.461678 (+ 859us) client_negotiation.cc:240] Negotiation successful
0618 22:27:01.461843 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":548,"threads_started":1}
I20260618 22:27:01.465262 13107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51d9c50e09940e6bf4cd7cc4360fe05" candidate_uuid: "fe6f8874615b456bbb65c66e009135f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0f878d5d97442cae587020545849ed" is_pre_election: true
I20260618 22:27:01.465999 13179 leader_election.cc:302] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fe6f8874615b456bbb65c66e009135f6; no voters: 9bb994c822f14df7a83e33284f452f51, 9f0f878d5d97442cae587020545849ed
I20260618 22:27:01.466410 13462 raft_consensus.cc:2726] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 22:27:01.466197 12951 catalog_manager.cc:5495] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f0f878d5d97442cae587020545849ed (127.10.242.193). New cstate: current_term: 1 leader_uuid: "9f0f878d5d97442cae587020545849ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } health_report { overall_health: UNKNOWN } } }
I20260618 22:27:01.521839 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:01.516066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42570 (local address 127.10.242.193:42231)
0618 22:27:01.516291 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:01.516297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:01.516319 (+ 22us) server_negotiation.cc:407] Connection header received
0618 22:27:01.516485 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:01.516490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:01.516574 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:01.516665 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:27:01.516958 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.518032 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.519397 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.520044 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.520165 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:01.520980 (+ 815us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:27:01.521134 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:27:01.521214 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:01.521587 (+ 373us) server_negotiation.cc:299] Negotiation successful
0618 22:27:01.521646 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260618 22:27:01.540740 13500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:01.536306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:42231 (local address 127.10.242.193:58535)
0618 22:27:01.536500 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:01.536520 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:01.536621 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:01.536897 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:01.536900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:01.536909 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:27:01.537073 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:01.537106 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.538112 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.538117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:01.539077 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:01.539088 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.539741 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.539747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:01.539860 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:01.539893 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:27:01.540208 (+ 315us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:27:01.540219 (+ 11us) client_negotiation.cc:769] Sending connection context
0618 22:27:01.540290 (+ 71us) client_negotiation.cc:240] Negotiation successful
0618 22:27:01.540465 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":23}
I20260618 22:27:01.540750 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:01.536454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:58535 (local address 127.10.242.193:42231)
0618 22:27:01.536609 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:01.536614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:01.536633 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:27:01.536711 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:01.536716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:01.536801 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:01.536889 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:27:01.537231 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.537993 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.539197 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:01.539612 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:01.539711 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:01.540017 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:27:01.540127 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:27:01.540195 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:01.540309 (+ 114us) server_negotiation.cc:299] Negotiation successful
0618 22:27:01.540413 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":36}
I20260618 22:27:01.549070 13386 raft_consensus.cc:1261] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51 [term 1 FOLLOWER]: Refusing update from remote peer 9f0f878d5d97442cae587020545849ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:01.549364 13246 raft_consensus.cc:1261] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Refusing update from remote peer 9f0f878d5d97442cae587020545849ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:01.549647 13490 consensus_queue.cc:1035] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:01.550024 13463 consensus_queue.cc:1035] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 22:27:01.575948 13483 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260618 22:27:03.180658 12950 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:03.180879 12950 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:03.184768 12950 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9: Sending DeleteTablet for 3 replicas of tablet e51d9c50e09940e6bf4cd7cc4360fe05
I20260618 22:27:03.185590 13088 tablet_service.cc:1514] Processing DeleteTablet for tablet e51d9c50e09940e6bf4cd7cc4360fe05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:03 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:42548
I20260618 22:27:03.185642 13227 tablet_service.cc:1514] Processing DeleteTablet for tablet e51d9c50e09940e6bf4cd7cc4360fe05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:03 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:35792
I20260618 22:27:03.185758 13365 tablet_service.cc:1514] Processing DeleteTablet for tablet e51d9c50e09940e6bf4cd7cc4360fe05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:03 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:55708
I20260618 22:27:03.186255 13534 tablet_replica.cc:330] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6: stopping tablet replica
I20260618 22:27:03.186424 13535 tablet_replica.cc:330] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed: stopping tablet replica
I20260618 22:27:03.186673 13535 raft_consensus.cc:2229] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:27:03.187258 13535 raft_consensus.cc:2258] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:03.188266 13535 ts_tablet_manager.cc:1905] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:03.186517 13534 raft_consensus.cc:2229] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:03.190968 13534 raft_consensus.cc:2258] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:03.191697 13536 tablet_replica.cc:330] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51: stopping tablet replica
I20260618 22:27:03.191900 13536 raft_consensus.cc:2229] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:03.192169 13536 raft_consensus.cc:2258] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:03.193219 13536 ts_tablet_manager.cc:1905] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:03.194720 13535 ts_tablet_manager.cc:1918] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260618 22:27:03.194900 13535 log.cc:1196] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/e51d9c50e09940e6bf4cd7cc4360fe05
I20260618 22:27:03.195098 13534 ts_tablet_manager.cc:1905] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:03.195353 13535 ts_tablet_manager.cc:1939] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9f0f878d5d97442cae587020545849ed: Deleting consensus metadata
I20260618 22:27:03.196756 12936 catalog_manager.cc:4841] TS 9f0f878d5d97442cae587020545849ed (127.10.242.193:42231): tablet e51d9c50e09940e6bf4cd7cc4360fe05 (table test-table [id=75b30de20ab94926a3f44b436e700eff]) successfully deleted
I20260618 22:27:03.199359 13536 ts_tablet_manager.cc:1918] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260618 22:27:03.199507 13536 log.cc:1196] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/e51d9c50e09940e6bf4cd7cc4360fe05
I20260618 22:27:03.199903 13536 ts_tablet_manager.cc:1939] T e51d9c50e09940e6bf4cd7cc4360fe05 P 9bb994c822f14df7a83e33284f452f51: Deleting consensus metadata
I20260618 22:27:03.200948 12936 catalog_manager.cc:4841] TS 9bb994c822f14df7a83e33284f452f51 (127.10.242.195:40331): tablet e51d9c50e09940e6bf4cd7cc4360fe05 (table test-table [id=75b30de20ab94926a3f44b436e700eff]) successfully deleted
I20260618 22:27:03.201169 13534 ts_tablet_manager.cc:1918] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260618 22:27:03.201294 13534 log.cc:1196] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/e51d9c50e09940e6bf4cd7cc4360fe05
I20260618 22:27:03.201622 13534 ts_tablet_manager.cc:1939] T e51d9c50e09940e6bf4cd7cc4360fe05 P fe6f8874615b456bbb65c66e009135f6: Deleting consensus metadata
I20260618 22:27:03.202425 12935 catalog_manager.cc:4841] TS fe6f8874615b456bbb65c66e009135f6 (127.10.242.194:42087): tablet e51d9c50e09940e6bf4cd7cc4360fe05 (table test-table [id=75b30de20ab94926a3f44b436e700eff]) successfully deleted
Jun 18 22:27:03 dist-test-slave-0956 krb5kdc[12906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 18 22:27:03 dist-test-slave-0956 krb5kdc[12906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:03.237748 13544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:03.218151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43968 (local address 127.10.242.254:37899)
0618 22:27:03.218739 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:03.218746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:03.218769 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:27:03.218881 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:03.218884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:03.218975 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:03.219063 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:03.220419 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:03.221283 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:03.222614 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:03.223010 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:03.223116 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:03.232741 (+ 9625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:03.232747 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:03.232751 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:03.232793 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:03.235454 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:03.236154 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:03.236158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:03.236160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:03.236285 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:03.236577 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:03.236580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:03.236583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:03.236838 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:03.237032 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:03.237281 (+ 249us) server_negotiation.cc:299] Negotiation successful
0618 22:27:03.237446 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":297,"threads_started":1}
I20260618 22:27:03.241046 12950 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:27:03.241636 12950 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:27:03.248127 13227 tablet_service.cc:1467] Processing CreateTablet for tablet 11896749c54542619d0908eb5179fb72 (DEFAULT_TABLE table=test-table [id=0c66832e93a54d30a037095dd8885352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:03.248119 13088 tablet_service.cc:1467] Processing CreateTablet for tablet 11896749c54542619d0908eb5179fb72 (DEFAULT_TABLE table=test-table [id=0c66832e93a54d30a037095dd8885352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:03.248450 13365 tablet_service.cc:1467] Processing CreateTablet for tablet 11896749c54542619d0908eb5179fb72 (DEFAULT_TABLE table=test-table [id=0c66832e93a54d30a037095dd8885352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:03.248607 13227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11896749c54542619d0908eb5179fb72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:03.248628 13088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11896749c54542619d0908eb5179fb72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:03.248859 13365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11896749c54542619d0908eb5179fb72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:03.255190 13545 tablet_bootstrap.cc:492] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed: Bootstrap starting.
I20260618 22:27:03.256744 13545 tablet_bootstrap.cc:654] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:03.257737 13547 tablet_bootstrap.cc:492] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6: Bootstrap starting.
I20260618 22:27:03.258364 13546 tablet_bootstrap.cc:492] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51: Bootstrap starting.
I20260618 22:27:03.259827 13545 tablet_bootstrap.cc:492] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed: No bootstrap required, opened a new log
I20260618 22:27:03.259965 13545 ts_tablet_manager.cc:1397] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260618 22:27:03.260223 13546 tablet_bootstrap.cc:654] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:03.260591 13547 tablet_bootstrap.cc:654] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:03.260622 13545 raft_consensus.cc:348] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:03.260841 13545 raft_consensus.cc:374] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:03.260932 13545 raft_consensus.cc:729] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0f878d5d97442cae587020545849ed, State: Initialized, Role: FOLLOWER
I20260618 22:27:03.261191 13545 consensus_queue.cc:260] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:03.261747 13545 ts_tablet_manager.cc:1428] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 22:27:03.262555 13546 tablet_bootstrap.cc:492] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51: No bootstrap required, opened a new log
I20260618 22:27:03.262686 13546 ts_tablet_manager.cc:1397] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260618 22:27:03.263341 13546 raft_consensus.cc:348] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:03.263531 13546 raft_consensus.cc:374] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:03.263617 13546 raft_consensus.cc:729] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bb994c822f14df7a83e33284f452f51, State: Initialized, Role: FOLLOWER
I20260618 22:27:03.263847 13546 consensus_queue.cc:260] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:03.265038 13546 ts_tablet_manager.cc:1428] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260618 22:27:03.266546 13547 tablet_bootstrap.cc:492] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6: No bootstrap required, opened a new log
I20260618 22:27:03.266712 13547 ts_tablet_manager.cc:1397] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260618 22:27:03.267536 13547 raft_consensus.cc:348] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:03.267845 13547 raft_consensus.cc:374] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:03.267962 13547 raft_consensus.cc:729] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe6f8874615b456bbb65c66e009135f6, State: Initialized, Role: FOLLOWER
I20260618 22:27:03.268222 13547 consensus_queue.cc:260] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:03.269019 13547 ts_tablet_manager.cc:1428] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 22:27:03.367523 13551 raft_consensus.cc:482] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:27:03.367740 13551 raft_consensus.cc:504] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:03.368384 13551 leader_election.cc:288] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f0f878d5d97442cae587020545849ed (127.10.242.193:42231), fe6f8874615b456bbb65c66e009135f6 (127.10.242.194:42087)
I20260618 22:27:03.368994 13247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11896749c54542619d0908eb5179fb72" candidate_uuid: "9bb994c822f14df7a83e33284f452f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6f8874615b456bbb65c66e009135f6" is_pre_election: true
I20260618 22:27:03.369024 13107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11896749c54542619d0908eb5179fb72" candidate_uuid: "9bb994c822f14df7a83e33284f452f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0f878d5d97442cae587020545849ed" is_pre_election: true
I20260618 22:27:03.369230 13247 raft_consensus.cc:2454] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bb994c822f14df7a83e33284f452f51 in term 0.
I20260618 22:27:03.369319 13107 raft_consensus.cc:2454] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bb994c822f14df7a83e33284f452f51 in term 0.
I20260618 22:27:03.369683 13317 leader_election.cc:302] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bb994c822f14df7a83e33284f452f51, fe6f8874615b456bbb65c66e009135f6; no voters:
I20260618 22:27:03.369946 13551 raft_consensus.cc:2781] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:27:03.370067 13551 raft_consensus.cc:482] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:27:03.370157 13551 raft_consensus.cc:3037] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:03.372192 13551 raft_consensus.cc:504] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:03.372741 13551 leader_election.cc:288] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [CANDIDATE]: Term 1 election: Requested vote from peers 9f0f878d5d97442cae587020545849ed (127.10.242.193:42231), fe6f8874615b456bbb65c66e009135f6 (127.10.242.194:42087)
I20260618 22:27:03.373271 13107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11896749c54542619d0908eb5179fb72" candidate_uuid: "9bb994c822f14df7a83e33284f452f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0f878d5d97442cae587020545849ed"
I20260618 22:27:03.373339 13247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11896749c54542619d0908eb5179fb72" candidate_uuid: "9bb994c822f14df7a83e33284f452f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6f8874615b456bbb65c66e009135f6"
I20260618 22:27:03.373471 13107 raft_consensus.cc:3037] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:03.373531 13247 raft_consensus.cc:3037] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:03.375567 13107 raft_consensus.cc:2454] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bb994c822f14df7a83e33284f452f51 in term 1.
I20260618 22:27:03.375661 13247 raft_consensus.cc:2454] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bb994c822f14df7a83e33284f452f51 in term 1.
I20260618 22:27:03.375998 13318 leader_election.cc:302] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bb994c822f14df7a83e33284f452f51, 9f0f878d5d97442cae587020545849ed; no voters:
I20260618 22:27:03.376395 13551 raft_consensus.cc:2781] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:03.376816 13551 raft_consensus.cc:686] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [term 1 LEADER]: Becoming Leader. State: Replica: 9bb994c822f14df7a83e33284f452f51, State: Running, Role: LEADER
I20260618 22:27:03.377116 13551 consensus_queue.cc:237] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:03.379621 12950 catalog_manager.cc:5495] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9bb994c822f14df7a83e33284f452f51 (127.10.242.195). New cstate: current_term: 1 leader_uuid: "9bb994c822f14df7a83e33284f452f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } health_report { overall_health: HEALTHY } } }
I20260618 22:27:03.423952 13556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:03.419113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55724 (local address 127.10.242.195:40331)
0618 22:27:03.419674 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:03.419680 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:03.419703 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:27:03.419764 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:03.419769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:03.419846 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:03.419928 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:27:03.420284 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:03.421238 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:03.422497 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:03.422905 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:03.422993 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:03.423331 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:27:03.423460 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:27:03.423523 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:03.423620 (+ 97us) server_negotiation.cc:299] Negotiation successful
0618 22:27:03.423679 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":276,"threads_started":1}
I20260618 22:27:03.432590 13247 raft_consensus.cc:1261] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Refusing update from remote peer 9bb994c822f14df7a83e33284f452f51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:03.432618 13107 raft_consensus.cc:1261] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed [term 1 FOLLOWER]: Refusing update from remote peer 9bb994c822f14df7a83e33284f452f51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:03.433161 13553 consensus_queue.cc:1035] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:03.433485 13551 consensus_queue.cc:1035] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:03.483284 12951 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:03.483564 12951 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:03.486518 12951 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9: Sending DeleteTablet for 3 replicas of tablet 11896749c54542619d0908eb5179fb72
I20260618 22:27:03.487326 13088 tablet_service.cc:1514] Processing DeleteTablet for tablet 11896749c54542619d0908eb5179fb72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:03 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:42548
I20260618 22:27:03.487402 13365 tablet_service.cc:1514] Processing DeleteTablet for tablet 11896749c54542619d0908eb5179fb72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:03 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:55708
I20260618 22:27:03.487461 13227 tablet_service.cc:1514] Processing DeleteTablet for tablet 11896749c54542619d0908eb5179fb72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:03 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:35792
I20260618 22:27:03.487653 13536 tablet_replica.cc:330] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51: stopping tablet replica
I20260618 22:27:03.487722 13534 tablet_replica.cc:330] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6: stopping tablet replica
I20260618 22:27:03.487748 13535 tablet_replica.cc:330] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed: stopping tablet replica
I20260618 22:27:03.487844 13536 raft_consensus.cc:2229] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:27:03.487890 13534 raft_consensus.cc:2229] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:03.487941 13535 raft_consensus.cc:2229] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:03.488059 13534 raft_consensus.cc:2258] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:03.488122 13535 raft_consensus.cc:2258] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:03.488158 13536 raft_consensus.cc:2258] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:03.488757 12951 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 22:27:03.489187 13534 ts_tablet_manager.cc:1905] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:03.489344 13535 ts_tablet_manager.cc:1905] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 22:27:03.489427 12951 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:27:03.489437 13536 ts_tablet_manager.cc:1905] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:03.496289 13536 ts_tablet_manager.cc:1918] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:27:03.496560 13536 log.cc:1196] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/11896749c54542619d0908eb5179fb72
I20260618 22:27:03.497334 13536 ts_tablet_manager.cc:1939] T 11896749c54542619d0908eb5179fb72 P 9bb994c822f14df7a83e33284f452f51: Deleting consensus metadata
I20260618 22:27:03.497799 13227 tablet_service.cc:1467] Processing CreateTablet for tablet d3cb6ae2aa1e49b4b0d341849b509e7e (DEFAULT_TABLE table=test-table [id=30da54c5e77043b9a4479e06f6e38892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:03.497848 13365 tablet_service.cc:1467] Processing CreateTablet for tablet d3cb6ae2aa1e49b4b0d341849b509e7e (DEFAULT_TABLE table=test-table [id=30da54c5e77043b9a4479e06f6e38892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:03.498255 13365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3cb6ae2aa1e49b4b0d341849b509e7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:03.498255 13227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3cb6ae2aa1e49b4b0d341849b509e7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:03.498668 12936 catalog_manager.cc:4841] TS 9bb994c822f14df7a83e33284f452f51 (127.10.242.195:40331): tablet 11896749c54542619d0908eb5179fb72 (table test-table [id=0c66832e93a54d30a037095dd8885352]) successfully deleted
I20260618 22:27:03.499264 13535 ts_tablet_manager.cc:1918] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:27:03.499421 13535 log.cc:1196] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/11896749c54542619d0908eb5179fb72
I20260618 22:27:03.499851 13535 ts_tablet_manager.cc:1939] T 11896749c54542619d0908eb5179fb72 P 9f0f878d5d97442cae587020545849ed: Deleting consensus metadata
I20260618 22:27:03.501206 12936 catalog_manager.cc:4841] TS 9f0f878d5d97442cae587020545849ed (127.10.242.193:42231): tablet 11896749c54542619d0908eb5179fb72 (table test-table [id=0c66832e93a54d30a037095dd8885352]) successfully deleted
I20260618 22:27:03.501451 13088 tablet_service.cc:1467] Processing CreateTablet for tablet d3cb6ae2aa1e49b4b0d341849b509e7e (DEFAULT_TABLE table=test-table [id=30da54c5e77043b9a4479e06f6e38892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:03.501885 13088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3cb6ae2aa1e49b4b0d341849b509e7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:03.503893 13547 tablet_bootstrap.cc:492] T d3cb6ae2aa1e49b4b0d341849b509e7e P fe6f8874615b456bbb65c66e009135f6: Bootstrap starting.
I20260618 22:27:03.504041 13534 ts_tablet_manager.cc:1918] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:27:03.504174 13534 log.cc:1196] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/11896749c54542619d0908eb5179fb72
I20260618 22:27:03.504209 13546 tablet_bootstrap.cc:492] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51: Bootstrap starting.
I20260618 22:27:03.504596 13534 ts_tablet_manager.cc:1939] T 11896749c54542619d0908eb5179fb72 P fe6f8874615b456bbb65c66e009135f6: Deleting consensus metadata
I20260618 22:27:03.505630 13547 tablet_bootstrap.cc:654] T d3cb6ae2aa1e49b4b0d341849b509e7e P fe6f8874615b456bbb65c66e009135f6: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:03.505738 13546 tablet_bootstrap.cc:654] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:03.505903 12935 catalog_manager.cc:4841] TS fe6f8874615b456bbb65c66e009135f6 (127.10.242.194:42087): tablet 11896749c54542619d0908eb5179fb72 (table test-table [id=0c66832e93a54d30a037095dd8885352]) successfully deleted
I20260618 22:27:03.508772 13547 tablet_bootstrap.cc:492] T d3cb6ae2aa1e49b4b0d341849b509e7e P fe6f8874615b456bbb65c66e009135f6: No bootstrap required, opened a new log
I20260618 22:27:03.508941 13547 ts_tablet_manager.cc:1397] T d3cb6ae2aa1e49b4b0d341849b509e7e P fe6f8874615b456bbb65c66e009135f6: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260618 22:27:03.509374 13546 tablet_bootstrap.cc:492] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51: No bootstrap required, opened a new log
I20260618 22:27:03.509532 13546 ts_tablet_manager.cc:1397] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260618 22:27:03.509601 13547 raft_consensus.cc:348] T d3cb6ae2aa1e49b4b0d341849b509e7e P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:03.509824 13547 raft_consensus.cc:374] T d3cb6ae2aa1e49b4b0d341849b509e7e P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:03.509898 13547 raft_consensus.cc:729] T d3cb6ae2aa1e49b4b0d341849b509e7e P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe6f8874615b456bbb65c66e009135f6, State: Initialized, Role: FOLLOWER
I20260618 22:27:03.510169 13545 tablet_bootstrap.cc:492] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9f0f878d5d97442cae587020545849ed: Bootstrap starting.
I20260618 22:27:03.510221 13546 raft_consensus.cc:348] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:03.510486 13546 raft_consensus.cc:374] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:03.510566 13546 raft_consensus.cc:729] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bb994c822f14df7a83e33284f452f51, State: Initialized, Role: FOLLOWER
I20260618 22:27:03.510774 13546 consensus_queue.cc:260] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:03.511224 13546 ts_tablet_manager.cc:1428] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 22:27:03.511364 13547 consensus_queue.cc:260] T d3cb6ae2aa1e49b4b0d341849b509e7e P fe6f8874615b456bbb65c66e009135f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:03.511711 13545 tablet_bootstrap.cc:654] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9f0f878d5d97442cae587020545849ed: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:03.512488 13547 ts_tablet_manager.cc:1428] T d3cb6ae2aa1e49b4b0d341849b509e7e P fe6f8874615b456bbb65c66e009135f6: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260618 22:27:03.514900 13545 tablet_bootstrap.cc:492] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9f0f878d5d97442cae587020545849ed: No bootstrap required, opened a new log
I20260618 22:27:03.515136 13545 ts_tablet_manager.cc:1397] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9f0f878d5d97442cae587020545849ed: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.002s
I20260618 22:27:03.516057 13545 raft_consensus.cc:348] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:03.516299 13545 raft_consensus.cc:374] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:03.516382 13545 raft_consensus.cc:729] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0f878d5d97442cae587020545849ed, State: Initialized, Role: FOLLOWER
I20260618 22:27:03.516616 13545 consensus_queue.cc:260] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9f0f878d5d97442cae587020545849ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:03.517194 13545 ts_tablet_manager.cc:1428] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9f0f878d5d97442cae587020545849ed: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 22:27:03.551429 13551 raft_consensus.cc:482] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:27:03.551627 13551 raft_consensus.cc:504] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:03.552318 13551 leader_election.cc:288] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f0f878d5d97442cae587020545849ed (127.10.242.193:42231), fe6f8874615b456bbb65c66e009135f6 (127.10.242.194:42087)
I20260618 22:27:03.552865 13107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3cb6ae2aa1e49b4b0d341849b509e7e" candidate_uuid: "9bb994c822f14df7a83e33284f452f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0f878d5d97442cae587020545849ed" is_pre_election: true
I20260618 22:27:03.552923 13247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3cb6ae2aa1e49b4b0d341849b509e7e" candidate_uuid: "9bb994c822f14df7a83e33284f452f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6f8874615b456bbb65c66e009135f6" is_pre_election: true
I20260618 22:27:03.553155 13107 raft_consensus.cc:2454] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bb994c822f14df7a83e33284f452f51 in term 0.
I20260618 22:27:03.553160 13247 raft_consensus.cc:2454] T d3cb6ae2aa1e49b4b0d341849b509e7e P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bb994c822f14df7a83e33284f452f51 in term 0.
I20260618 22:27:03.553599 13318 leader_election.cc:302] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bb994c822f14df7a83e33284f452f51, 9f0f878d5d97442cae587020545849ed; no voters:
I20260618 22:27:03.553960 13551 raft_consensus.cc:2781] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:27:03.554078 13551 raft_consensus.cc:482] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:27:03.554162 13551 raft_consensus.cc:3037] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:03.577512 13551 raft_consensus.cc:504] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:03.578299 13551 leader_election.cc:288] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [CANDIDATE]: Term 1 election: Requested vote from peers 9f0f878d5d97442cae587020545849ed (127.10.242.193:42231), fe6f8874615b456bbb65c66e009135f6 (127.10.242.194:42087)
I20260618 22:27:03.578948 13107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3cb6ae2aa1e49b4b0d341849b509e7e" candidate_uuid: "9bb994c822f14df7a83e33284f452f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0f878d5d97442cae587020545849ed"
I20260618 22:27:03.578971 13247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3cb6ae2aa1e49b4b0d341849b509e7e" candidate_uuid: "9bb994c822f14df7a83e33284f452f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6f8874615b456bbb65c66e009135f6"
I20260618 22:27:03.579188 13107 raft_consensus.cc:3037] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9f0f878d5d97442cae587020545849ed [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:03.579200 13247 raft_consensus.cc:3037] T d3cb6ae2aa1e49b4b0d341849b509e7e P fe6f8874615b456bbb65c66e009135f6 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:03.661594 13247 raft_consensus.cc:2454] T d3cb6ae2aa1e49b4b0d341849b509e7e P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bb994c822f14df7a83e33284f452f51 in term 1.
I20260618 22:27:03.661592 13107 raft_consensus.cc:2454] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9f0f878d5d97442cae587020545849ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bb994c822f14df7a83e33284f452f51 in term 1.
I20260618 22:27:03.662297 13318 leader_election.cc:302] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 9bb994c822f14df7a83e33284f452f51, 9f0f878d5d97442cae587020545849ed, fe6f8874615b456bbb65c66e009135f6; no voters:
I20260618 22:27:03.662683 13551 raft_consensus.cc:2781] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:03.662835 13551 raft_consensus.cc:686] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [term 1 LEADER]: Becoming Leader. State: Replica: 9bb994c822f14df7a83e33284f452f51, State: Running, Role: LEADER
I20260618 22:27:03.663122 13551 consensus_queue.cc:237] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } }
I20260618 22:27:03.665714 12949 catalog_manager.cc:5495] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9bb994c822f14df7a83e33284f452f51 (127.10.242.195). New cstate: current_term: 1 leader_uuid: "9bb994c822f14df7a83e33284f452f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bb994c822f14df7a83e33284f452f51" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40331 } health_report { overall_health: HEALTHY } } }
I20260618 22:27:03.713855 13247 raft_consensus.cc:1261] T d3cb6ae2aa1e49b4b0d341849b509e7e P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Refusing update from remote peer 9bb994c822f14df7a83e33284f452f51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:03.713874 13107 raft_consensus.cc:1261] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9f0f878d5d97442cae587020545849ed [term 1 FOLLOWER]: Refusing update from remote peer 9bb994c822f14df7a83e33284f452f51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:03.714339 13551 consensus_queue.cc:1035] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe6f8874615b456bbb65c66e009135f6" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:03.714649 13553 consensus_queue.cc:1035] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f0f878d5d97442cae587020545849ed" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:03.725138 13515 mvcc.cc:205] Tried to move back new op lower bound from 7298341370727632896 to 7298341370526474240. Current Snapshot: MvccSnapshot[applied={T|T < 7298341370727632896}]
W20260618 22:27:03.732959 13518 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260618 22:27:05.312695 12950 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:05.312907 12950 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:05.316821 12950 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 079be3f013834f218fd9d5063fe112d9: Sending DeleteTablet for 3 replicas of tablet d3cb6ae2aa1e49b4b0d341849b509e7e
I20260618 22:27:05.317683 13088 tablet_service.cc:1514] Processing DeleteTablet for tablet d3cb6ae2aa1e49b4b0d341849b509e7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:05 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:42548
I20260618 22:27:05.317744 13227 tablet_service.cc:1514] Processing DeleteTablet for tablet d3cb6ae2aa1e49b4b0d341849b509e7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:05 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:35792
I20260618 22:27:05.317770 13365 tablet_service.cc:1514] Processing DeleteTablet for tablet d3cb6ae2aa1e49b4b0d341849b509e7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:05 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:55708
I20260618 22:27:05.318636 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 13022
I20260618 22:27:05.355890 13589 tablet_replica.cc:330] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9f0f878d5d97442cae587020545849ed: stopping tablet replica
I20260618 22:27:05.356465 13590 tablet_replica.cc:330] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51: stopping tablet replica
I20260618 22:27:05.356783 13590 raft_consensus.cc:2229] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:27:05.357415 13590 raft_consensus.cc:2258] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:05.357465 13591 tablet_replica.cc:330] T d3cb6ae2aa1e49b4b0d341849b509e7e P fe6f8874615b456bbb65c66e009135f6: stopping tablet replica
I20260618 22:27:05.357051 13589 raft_consensus.cc:2229] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9f0f878d5d97442cae587020545849ed [term 1 FOLLOWER]: Raft consensus shutting down.
Jun 18 22:27:05 dist-test-slave-0956 krb5kdc[12906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:27:05.357645 13591 raft_consensus.cc:2229] T d3cb6ae2aa1e49b4b0d341849b509e7e P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:05.357801 13589 raft_consensus.cc:2258] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9f0f878d5d97442cae587020545849ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:05.357872 13591 raft_consensus.cc:2258] T d3cb6ae2aa1e49b4b0d341849b509e7e P fe6f8874615b456bbb65c66e009135f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:05.358632 13590 ts_tablet_manager.cc:1905] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:05.358817 13591 ts_tablet_manager.cc:1905] T d3cb6ae2aa1e49b4b0d341849b509e7e P fe6f8874615b456bbb65c66e009135f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:05.360864 13589 ts_tablet_manager.cc:1905] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9f0f878d5d97442cae587020545849ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:05.365443 13593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:05.320219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42584 (local address 127.10.242.193:42231)
0618 22:27:05.344525 (+ 24306us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:05.344532 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:05.344558 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:27:05.344635 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:05.344641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:05.344740 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:05.344856 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:05.347151 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:05.349064 (+ 1913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:05.351271 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:05.351960 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:05.352084 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:05.359201 (+ 7117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:05.359222 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:05.359242 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:05.359291 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:05.362828 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:05.363647 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:05.363654 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:05.363657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:05.363789 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:05.364087 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:05.364091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:05.364093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:05.364431 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:05.364638 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:05.364858 (+ 220us) server_negotiation.cc:299] Negotiation successful
0618 22:27:05.365061 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24136,"thread_start_us":28885,"threads_started":1}
W20260618 22:27:05.366079 13153 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42584
W20260618 22:27:05.366669 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 13022: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:27:05.366838 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 13022
I20260618 22:27:05.435636 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 13162
W20260618 22:27:05.435586 12936 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.242.193:42231 (error 108)
I20260618 22:27:05.437716 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:05.436804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:42231 (local address 127.0.0.1:42598)
0618 22:27:05.437339 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:05.437510 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:42231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":301,"threads_started":1}
W20260618 22:27:05.437932 12936 catalog_manager.cc:4584] TS 9f0f878d5d97442cae587020545849ed (127.10.242.193:42231): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d3cb6ae2aa1e49b4b0d341849b509e7e: Network error: Client connection negotiation failed: client connection to 127.10.242.193:42231: connect: Connection refused (error 111)
Jun 18 22:27:05 dist-test-slave-0956 krb5kdc[12906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.194@KRBTEST.COM
I20260618 22:27:05.451402 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:05.436317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35806 (local address 127.10.242.194:42087)
0618 22:27:05.437380 (+ 1063us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:05.437384 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:05.437402 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:27:05.437513 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:05.437516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:05.437575 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:05.437659 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:05.438896 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:05.439644 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:05.440866 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:05.441315 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:05.441407 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:05.445636 (+ 4229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:05.445653 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:05.445669 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:05.445715 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:05.449030 (+ 3315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:05.449706 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:05.449713 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:05.449716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:05.449866 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:05.450174 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:05.450177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:05.450179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:05.450505 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:05.450720 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:05.450934 (+ 214us) server_negotiation.cc:299] Negotiation successful
0618 22:27:05.451141 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":965,"thread_start_us":849,"threads_started":1}
W20260618 22:27:05.452106 13292 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35806
W20260618 22:27:05.452739 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 13162: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:27:05.452893 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 13162
I20260618 22:27:05.491660 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:05.491189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:42231 (local address 127.0.0.1:42600)
0618 22:27:05.491405 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:05.491511 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:42231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260618 22:27:05.492729 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:05.492377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:42231 (local address 127.0.0.1:42610)
0618 22:27:05.492500 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:05.492589 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:42231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260618 22:27:05.518343 13590 ts_tablet_manager.cc:1918] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260618 22:27:05.518512 13590 log.cc:1196] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/d3cb6ae2aa1e49b4b0d341849b509e7e
I20260618 22:27:05.518877 13590 ts_tablet_manager.cc:1939] T d3cb6ae2aa1e49b4b0d341849b509e7e P 9bb994c822f14df7a83e33284f452f51: Deleting consensus metadata
I20260618 22:27:05.524133 12936 catalog_manager.cc:4841] TS 9bb994c822f14df7a83e33284f452f51 (127.10.242.195:40331): tablet d3cb6ae2aa1e49b4b0d341849b509e7e (table test-table [id=30da54c5e77043b9a4479e06f6e38892]) successfully deleted
I20260618 22:27:05.530740 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 13301
I20260618 22:27:05.531539 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:05.531131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:42087 (local address 127.0.0.1:35814)
0618 22:27:05.531297 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:05.531381 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.194:42087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
Jun 18 22:27:05 dist-test-slave-0956 krb5kdc[12906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.195@KRBTEST.COM
I20260618 22:27:05.545367 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:05.531426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55730 (local address 127.10.242.195:40331)
0618 22:27:05.531885 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:05.531890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:05.531903 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:27:05.531951 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:05.531956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:05.532016 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:05.532105 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:05.533409 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:05.534139 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:05.535313 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:05.535720 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:05.535827 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:05.539838 (+ 4011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:05.539853 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:05.539869 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:05.539914 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:05.542731 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:05.543384 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:05.543391 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:05.543394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:05.543526 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:05.543872 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:05.543875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:05.543877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:05.544211 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:05.544499 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:05.544780 (+ 281us) server_negotiation.cc:299] Negotiation successful
0618 22:27:05.544994 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":302,"threads_started":1}
W20260618 22:27:05.545996 13430 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55730
W20260618 22:27:05.546599 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 13301: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:27:05.546736 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 13301
I20260618 22:27:05.553771 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:05.553315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:42231 (local address 127.0.0.1:42622)
0618 22:27:05.553498 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:05.553581 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:42231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260618 22:27:05.554872 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:05.554499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:42231 (local address 127.0.0.1:42626)
0618 22:27:05.554670 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:05.554745 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:42231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260618 22:27:05.563906 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 12919
I20260618 22:27:05.564225 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:05.563707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:42087 (local address 127.0.0.1:35828)
0618 22:27:05.563920 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:05.564040 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.194:42087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260618 22:27:05.565128 12991 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:05.565485 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:05.565058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:42087 (local address 127.0.0.1:35844)
0618 22:27:05.565237 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:05.565328 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.194:42087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260618 22:27:05.678097 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:05.677522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:42231 (local address 127.0.0.1:42628)
0618 22:27:05.677758 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:05.677858 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:42231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260618 22:27:05.678359 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 12919
2026-06-18T22:27:05Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9102 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:27:05 dist-test-slave-0956 krb5kdc[13602](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:27:05 dist-test-slave-0956 krb5kdc[13602](info): set up 2 sockets
Jun 18 22:27:05 dist-test-slave-0956 krb5kdc[13602](info): commencing operation
krb5kdc: starting...
W20260618 22:27:06.673130 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.953s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:27:06 dist-test-slave-0956 krb5kdc[13602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:27:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:27:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:27:06.853143 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:35001
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:44783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:35001
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:07.169416 13615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:07.169705 13615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:07.169828 13615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:07.176877 13615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:27:07.176990 13615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:07.177037 13615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:07.177129 13615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:27:07.177189 13615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:27:07.187036 13615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:44783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:35001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:35001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:07.188365 13615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:07.189802 13615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:07.201644 13623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:07.201740 13620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:07.201722 13621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:07.202384 13615 server_base.cc:1034] running on GCE node
I20260618 22:27:07.203262 13615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:07.204782 13615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:07.205999 13615 hybrid_clock.cc:648] HybridClock initialized: now 1781821627205931 us; error 52 us; skew 500 ppm
I20260618 22:27:07.206362 13615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:07 dist-test-slave-0956 krb5kdc[13602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821627, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:07.209496 13615 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:27:07.210943 13615 webserver.cc:460] Webserver started at http://127.10.242.254:35349/ using document root <none> and password file <none>
I20260618 22:27:07.211634 13615 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:07.211746 13615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:07.212033 13615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:07.214942 13615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "d500bb01d04647f29cb6d96edd8333cf"
format_stamp: "Formatted at 2026-06-18 22:27:07 on dist-test-slave-0956"
server_key: "0c9c84d40053b1d228e366e2506499b4"
server_key_iv: "6d94eaa9a8128c49e0faf2bc3540e5d2"
server_key_version: "encryptionkey@0"
I20260618 22:27:07.215667 13615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "d500bb01d04647f29cb6d96edd8333cf"
format_stamp: "Formatted at 2026-06-18 22:27:07 on dist-test-slave-0956"
server_key: "0c9c84d40053b1d228e366e2506499b4"
server_key_iv: "6d94eaa9a8128c49e0faf2bc3540e5d2"
server_key_version: "encryptionkey@0"
I20260618 22:27:07.221415 13615 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260618 22:27:07.224829 13630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:07.226256 13615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:27:07.226423 13615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "d500bb01d04647f29cb6d96edd8333cf"
format_stamp: "Formatted at 2026-06-18 22:27:07 on dist-test-slave-0956"
server_key: "0c9c84d40053b1d228e366e2506499b4"
server_key_iv: "6d94eaa9a8128c49e0faf2bc3540e5d2"
server_key_version: "encryptionkey@0"
I20260618 22:27:07.226626 13615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:07.255681 13615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:07.260015 13615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:07.260282 13615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:07.277664 13615 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:35001
I20260618 22:27:07.277685 13681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:35001 every 8 connection(s)
I20260618 22:27:07.279042 13615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:27:07.280493 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 13615
I20260618 22:27:07.280692 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:27:07.281505 11211 external_mini_cluster.cc:1428] Setting key 26b6aefe2a799bf802c94cc87a4eb39e
I20260618 22:27:07.283730 13682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 18 22:27:07 dist-test-slave-0956 krb5kdc[13602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:07.296469 13682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf: Bootstrap starting.
I20260618 22:27:07.300272 13682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:07.301472 13682 log.cc:824] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:07.304715 13682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf: No bootstrap required, opened a new log
I20260618 22:27:07.305121 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:07.283559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54002 (local address 127.10.242.254:35001)
0618 22:27:07.284338 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:07.284353 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:07.284399 (+ 46us) server_negotiation.cc:407] Connection header received
0618 22:27:07.286508 (+ 2109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:07.286536 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:07.287065 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:07.287512 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:07.288822 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:07.290841 (+ 2019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:07.292183 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:07.293210 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:07.293387 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:07.297270 (+ 3883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:07.297280 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:07.297300 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:07.297343 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:07.300795 (+ 3452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:07.301566 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:07.301575 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:07.301583 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:07.301780 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:07.302205 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:07.302209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:07.302211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:07.302724 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:07.302986 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:07.303381 (+ 395us) server_negotiation.cc:299] Negotiation successful
0618 22:27:07.303783 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":342,"threads_started":1}
I20260618 22:27:07.311946 13682 raft_consensus.cc:348] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d500bb01d04647f29cb6d96edd8333cf" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35001 } }
I20260618 22:27:07.312278 13682 raft_consensus.cc:374] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:07.312351 13682 raft_consensus.cc:729] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d500bb01d04647f29cb6d96edd8333cf, State: Initialized, Role: FOLLOWER
I20260618 22:27:07.312852 13682 consensus_queue.cc:260] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d500bb01d04647f29cb6d96edd8333cf" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35001 } }
I20260618 22:27:07.313041 13682 raft_consensus.cc:388] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:27:07.313169 13682 raft_consensus.cc:482] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:27:07.313344 13682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:07.315608 13682 raft_consensus.cc:504] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d500bb01d04647f29cb6d96edd8333cf" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35001 } }
I20260618 22:27:07.316100 13682 leader_election.cc:302] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d500bb01d04647f29cb6d96edd8333cf; no voters:
I20260618 22:27:07.316675 13682 leader_election.cc:288] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:27:07.316793 13687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:07.317404 13687 raft_consensus.cc:686] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf [term 1 LEADER]: Becoming Leader. State: Replica: d500bb01d04647f29cb6d96edd8333cf, State: Running, Role: LEADER
I20260618 22:27:07.317893 13687 consensus_queue.cc:237] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d500bb01d04647f29cb6d96edd8333cf" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35001 } }
I20260618 22:27:07.318650 13682 sys_catalog.cc:564] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:27:07.320276 13688 sys_catalog.cc:455] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d500bb01d04647f29cb6d96edd8333cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d500bb01d04647f29cb6d96edd8333cf" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35001 } } }
I20260618 22:27:07.320853 13689 sys_catalog.cc:455] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader d500bb01d04647f29cb6d96edd8333cf. Latest consensus state: current_term: 1 leader_uuid: "d500bb01d04647f29cb6d96edd8333cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d500bb01d04647f29cb6d96edd8333cf" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35001 } } }
I20260618 22:27:07.321566 13688 sys_catalog.cc:458] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:07.321532 13689 sys_catalog.cc:458] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:07.323328 13693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:27:07.327883 13693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:27:07.335999 13693 catalog_manager.cc:1347] Generated new cluster ID: 82eb7db6543540ae86b51f77e0ade998
I20260618 22:27:07.336115 13693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:27:07.349912 13693 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:27:07.351320 13693 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:27:07.363521 13693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d500bb01d04647f29cb6d96edd8333cf: Generated new TSK 0
I20260618 22:27:07.364431 13693 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
I20260618 22:27:07.406522 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:35001
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:44783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:07.725581 13708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:07.725893 13708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:07.725988 13708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:07.733506 13708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:07.733628 13708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:07.733744 13708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:27:07.743613 13708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:44783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:35001
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:07.744853 13708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:07.746183 13708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:07.759274 13713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:07.759351 13716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:07.759315 13714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:07.760159 13708 server_base.cc:1034] running on GCE node
I20260618 22:27:07.760800 13708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:07.761849 13708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:07.763070 13708 hybrid_clock.cc:648] HybridClock initialized: now 1781821627763014 us; error 48 us; skew 500 ppm
I20260618 22:27:07.763399 13708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:07 dist-test-slave-0956 krb5kdc[13602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821627, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:07.766861 13708 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:27:07.768460 13708 webserver.cc:460] Webserver started at http://127.10.242.193:46357/ using document root <none> and password file <none>
I20260618 22:27:07.769245 13708 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:07.769361 13708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:07.769687 13708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:07.772671 13708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "3409ac6465a441aa8ec787b07bcec3c1"
format_stamp: "Formatted at 2026-06-18 22:27:07 on dist-test-slave-0956"
server_key: "9efbcf77053411c458dabb3782f59a42"
server_key_iv: "d508e9d28ab7f2daa6e58ad666a77a78"
server_key_version: "encryptionkey@0"
I20260618 22:27:07.773499 13708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "3409ac6465a441aa8ec787b07bcec3c1"
format_stamp: "Formatted at 2026-06-18 22:27:07 on dist-test-slave-0956"
server_key: "9efbcf77053411c458dabb3782f59a42"
server_key_iv: "d508e9d28ab7f2daa6e58ad666a77a78"
server_key_version: "encryptionkey@0"
I20260618 22:27:07.778789 13708 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:27:07.782379 13723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:07.783787 13708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260618 22:27:07.783946 13708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "3409ac6465a441aa8ec787b07bcec3c1"
format_stamp: "Formatted at 2026-06-18 22:27:07 on dist-test-slave-0956"
server_key: "9efbcf77053411c458dabb3782f59a42"
server_key_iv: "d508e9d28ab7f2daa6e58ad666a77a78"
server_key_version: "encryptionkey@0"
I20260618 22:27:07.784140 13708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:07.817320 13708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:07.822170 13708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:07.822443 13708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:07.823417 13708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:07.824765 13708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:07.824844 13708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:07.824962 13708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:07.825027 13708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:07.854271 13708 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:33887
I20260618 22:27:07.854306 13835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:33887 every 8 connection(s)
I20260618 22:27:07.855552 13708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:27:07.864149 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 13708
I20260618 22:27:07.864367 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:27:07.865195 11211 external_mini_cluster.cc:1428] Setting key b4d1e55d2f1e3bee72f0911da8dfb068
Jun 18 22:27:07 dist-test-slave-0956 krb5kdc[13602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821627, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:07.877287 13838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:07.858475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:45529 (local address 127.10.242.254:35001)
0618 22:27:07.858925 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:07.858931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:07.860058 (+ 1127us) server_negotiation.cc:407] Connection header received
0618 22:27:07.862070 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:07.862075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:07.862158 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:07.862265 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:07.864089 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:07.865176 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:07.866853 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:07.867344 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:07.867468 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:07.871367 (+ 3899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:07.871373 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:07.871377 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:07.871427 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:07.874186 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:07.874992 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:07.874997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:07.875000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:07.875133 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:07.875511 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:07.875515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:07.875518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:07.875826 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:07.875963 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:07.876689 (+ 726us) server_negotiation.cc:299] Negotiation successful
0618 22:27:07.876901 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":216,"threads_started":1}
I20260618 22:27:07.878208 13839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:07.859042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35001 (local address 127.10.242.193:45529)
0618 22:27:07.859803 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:07.859855 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:07.861717 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:07.862560 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:07.862569 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:07.863197 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:07.863853 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:07.863883 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:07.865399 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:07.865404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:07.866714 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:07.866727 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:07.867574 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:07.867579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:07.867797 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:07.868572 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:07.868596 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:07.871098 (+ 2502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:07.874375 (+ 3277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:07.874384 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:07.874399 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:07.874868 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:07.875229 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:07.875232 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:07.875234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:07.875422 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:07.875930 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:07.875936 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:07.876333 (+ 397us) client_negotiation.cc:769] Sending connection context
0618 22:27:07.876580 (+ 247us) client_negotiation.cc:240] Negotiation successful
0618 22:27:07.877011 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":313,"threads_started":1}
I20260618 22:27:07.879762 13836 heartbeater.cc:347] Connected to a master server at 127.10.242.254:35001
I20260618 22:27:07.880066 13836 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:07.880779 13836 heartbeater.cc:511] Master 127.10.242.254:35001 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.242.194@KRBTEST.COM; defaulting to no policy
I20260618 22:27:07.882900 13647 ts_manager.cc:194] Registered new tserver with Master: 3409ac6465a441aa8ec787b07bcec3c1 (127.10.242.193:33887)
I20260618 22:27:07.884308 13647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:45529
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
I20260618 22:27:07.906342 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:35001
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:44783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:08.220752 13842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:08.221050 13842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:08.221207 13842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:08.228437 13842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:08.228569 13842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:08.228735 13842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:27:08.238536 13842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:44783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:35001
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:08.239840 13842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:08.241166 13842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:08.253793 13847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:08.253757 13848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:08.253728 13850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:08.254609 13842 server_base.cc:1034] running on GCE node
I20260618 22:27:08.255177 13842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:08.256109 13842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:08.257299 13842 hybrid_clock.cc:648] HybridClock initialized: now 1781821628257242 us; error 48 us; skew 500 ppm
I20260618 22:27:08.257624 13842 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:08 dist-test-slave-0956 krb5kdc[13602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821628, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:08.265216 13842 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:27:08.266817 13842 webserver.cc:460] Webserver started at http://127.10.242.194:35229/ using document root <none> and password file <none>
I20260618 22:27:08.267639 13842 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:08.267760 13842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:08.268039 13842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:08.271018 13842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "60a2e99f314246078ba785a154c6f4e4"
format_stamp: "Formatted at 2026-06-18 22:27:08 on dist-test-slave-0956"
server_key: "0a570069a0ec00620845533ee5b52ad3"
server_key_iv: "092e392f313ef09deb13d09f6fe3822a"
server_key_version: "encryptionkey@0"
I20260618 22:27:08.271826 13842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "60a2e99f314246078ba785a154c6f4e4"
format_stamp: "Formatted at 2026-06-18 22:27:08 on dist-test-slave-0956"
server_key: "0a570069a0ec00620845533ee5b52ad3"
server_key_iv: "092e392f313ef09deb13d09f6fe3822a"
server_key_version: "encryptionkey@0"
I20260618 22:27:08.277007 13842 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260618 22:27:08.280567 13857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:08.282060 13842 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:27:08.282199 13842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "60a2e99f314246078ba785a154c6f4e4"
format_stamp: "Formatted at 2026-06-18 22:27:08 on dist-test-slave-0956"
server_key: "0a570069a0ec00620845533ee5b52ad3"
server_key_iv: "092e392f313ef09deb13d09f6fe3822a"
server_key_version: "encryptionkey@0"
I20260618 22:27:08.282379 13842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:08.303061 13842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:08.307343 13842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:08.307619 13842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:08.308590 13842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:08.310022 13842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:08.310098 13842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:08.310180 13842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:08.310221 13842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:08.338577 13842 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:36647
I20260618 22:27:08.338665 13969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:36647 every 8 connection(s)
I20260618 22:27:08.339805 13842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:27:08.343209 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 13842
I20260618 22:27:08.343377 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:27:08.343925 11211 external_mini_cluster.cc:1428] Setting key 207d2a438ac62a48226f7914cf9f00f9
Jun 18 22:27:08 dist-test-slave-0956 krb5kdc[13602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821628, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.242.195@KRBTEST.COM; defaulting to no policy
I20260618 22:27:08.359453 13838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:08.342266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:37559 (local address 127.10.242.254:35001)
0618 22:27:08.342416 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:08.342420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:08.343736 (+ 1316us) server_negotiation.cc:407] Connection header received
0618 22:27:08.345733 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:08.345738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:08.345816 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:08.345896 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:08.347370 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:08.348075 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:08.349655 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:08.350230 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:08.350357 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:08.353940 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:08.353945 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:08.353948 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:08.353982 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:08.356463 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:08.357267 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:08.357271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:08.357273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:08.357400 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:08.357813 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:08.357817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:08.357819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:08.358079 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:08.358209 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:08.358898 (+ 689us) server_negotiation.cc:299] Negotiation successful
0618 22:27:08.359117 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260618 22:27:08.360447 13972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:08.342738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35001 (local address 127.10.242.194:37559)
0618 22:27:08.343530 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:08.343582 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:08.345467 (+ 1885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:08.346143 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:08.346152 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:08.346654 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:08.347209 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:08.347226 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:08.348218 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:08.348222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:08.349521 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:08.349533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:08.350467 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:08.350472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:08.350715 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:08.351379 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:08.351407 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:08.353697 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:08.356626 (+ 2929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:08.356636 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:08.356655 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:08.357145 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:08.357505 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:08.357510 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:08.357514 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:08.357701 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:08.358193 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:08.358201 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:08.358554 (+ 353us) client_negotiation.cc:769] Sending connection context
0618 22:27:08.358788 (+ 234us) client_negotiation.cc:240] Negotiation successful
0618 22:27:08.359338 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":285,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.195@KRBTEST.COM" created.
I20260618 22:27:08.362079 13970 heartbeater.cc:347] Connected to a master server at 127.10.242.254:35001
I20260618 22:27:08.362421 13970 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:08.363262 13970 heartbeater.cc:511] Master 127.10.242.254:35001 requested a full tablet report, sending...
I20260618 22:27:08.364784 13647 ts_manager.cc:194] Registered new tserver with Master: 60a2e99f314246078ba785a154c6f4e4 (127.10.242.194:36647)
I20260618 22:27:08.365661 13647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:37559
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
I20260618 22:27:08.382253 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:35001
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:44783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:08.695614 13975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:08.695899 13975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:08.696023 13975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:08.703233 13975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:08.703354 13975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:08.703503 13975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:27:08.713240 13975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:44783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:35001
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:08.714530 13975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:08.715780 13975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:08.728734 13980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:08.728735 13981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:08.728976 13983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:08.729483 13975 server_base.cc:1034] running on GCE node
I20260618 22:27:08.730062 13975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:08.730979 13975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:08.732173 13975 hybrid_clock.cc:648] HybridClock initialized: now 1781821628732113 us; error 52 us; skew 500 ppm
I20260618 22:27:08.732465 13975 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:08 dist-test-slave-0956 krb5kdc[13602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821628, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:08.735543 13975 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:27:08.737020 13975 webserver.cc:460] Webserver started at http://127.10.242.195:38649/ using document root <none> and password file <none>
I20260618 22:27:08.737769 13975 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:08.737876 13975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:08.738170 13975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:08.741024 13975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "f61966d2e6d74325ba60c01e7c3fb47a"
format_stamp: "Formatted at 2026-06-18 22:27:08 on dist-test-slave-0956"
server_key: "06814090eda810c186ee7cc7816fe175"
server_key_iv: "02880639e13c4ae737a01b5233e28bb1"
server_key_version: "encryptionkey@0"
I20260618 22:27:08.741828 13975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "f61966d2e6d74325ba60c01e7c3fb47a"
format_stamp: "Formatted at 2026-06-18 22:27:08 on dist-test-slave-0956"
server_key: "06814090eda810c186ee7cc7816fe175"
server_key_iv: "02880639e13c4ae737a01b5233e28bb1"
server_key_version: "encryptionkey@0"
I20260618 22:27:08.746812 13975 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:27:08.750339 13990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:08.751681 13975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:27:08.751832 13975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "f61966d2e6d74325ba60c01e7c3fb47a"
format_stamp: "Formatted at 2026-06-18 22:27:08 on dist-test-slave-0956"
server_key: "06814090eda810c186ee7cc7816fe175"
server_key_iv: "02880639e13c4ae737a01b5233e28bb1"
server_key_version: "encryptionkey@0"
I20260618 22:27:08.752027 13975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:08.776556 13975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:08.781149 13975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:08.781443 13975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:08.782469 13975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:08.783943 13975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:08.784024 13975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:08.784142 13975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:08.784194 13975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:08.814977 13975 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:40563
I20260618 22:27:08.815021 14102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:40563 every 8 connection(s)
I20260618 22:27:08.816231 13975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:27:08.819401 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 13975
I20260618 22:27:08.819537 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:27:08.820104 11211 external_mini_cluster.cc:1428] Setting key 2cab6abac7823aebacc456edab45cb5f
Jun 18 22:27:08 dist-test-slave-0956 krb5kdc[13602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821628, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:08.836158 13838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:08.818710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:50499 (local address 127.10.242.254:35001)
0618 22:27:08.818906 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:08.818912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:08.820415 (+ 1503us) server_negotiation.cc:407] Connection header received
0618 22:27:08.822720 (+ 2305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:08.822725 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:08.822809 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:08.822912 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:08.824632 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:08.825403 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:08.826921 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:08.827327 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:08.827420 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:08.830698 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:08.830703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:08.830707 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:08.830737 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:08.832993 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:08.833703 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:08.833708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:08.833711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:08.833850 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:08.834338 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:08.834342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:08.834344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:08.834667 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:08.834802 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:08.835638 (+ 836us) server_negotiation.cc:299] Negotiation successful
0618 22:27:08.835861 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:27:08.837073 14105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:08.819318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35001 (local address 127.10.242.195:50499)
0618 22:27:08.820225 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:08.820281 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:08.822459 (+ 2178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:08.823364 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:08.823372 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:08.823886 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:08.824477 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:08.824494 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:08.825551 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:08.825555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:08.826795 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:08.826807 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:08.827501 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:08.827506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:08.827696 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:08.828323 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:08.828344 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:08.830507 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:08.833153 (+ 2646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:08.833160 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:08.833174 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:08.833592 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:08.833967 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:08.833971 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:08.833974 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:08.834224 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:08.834789 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:08.834795 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:08.835189 (+ 394us) client_negotiation.cc:769] Sending connection context
0618 22:27:08.835491 (+ 302us) client_negotiation.cc:240] Negotiation successful
0618 22:27:08.835959 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":348,"threads_started":1}
I20260618 22:27:08.838284 14103 heartbeater.cc:347] Connected to a master server at 127.10.242.254:35001
I20260618 22:27:08.838546 14103 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:08.839175 14103 heartbeater.cc:511] Master 127.10.242.254:35001 requested a full tablet report, sending...
I20260618 22:27:08.840440 13647 ts_manager.cc:194] Registered new tserver with Master: f61966d2e6d74325ba60c01e7c3fb47a (127.10.242.195:40563)
I20260618 22:27:08.841264 13647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:50499
I20260618 22:27:08.848006 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260618 22:27:08.878515 13838 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:08.876160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54018 (local address 127.10.242.254:35001)
0618 22:27:08.876336 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:08.876340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:08.876823 (+ 483us) server_negotiation.cc:407] Connection header received
0618 22:27:08.876968 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:08.876973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:08.877043 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:08.877169 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:08.878288 (+ 1119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54018: BlockingRecv error: recv got EOF from 127.0.0.1:54018 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260618 22:27:08.881628 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 13708
W20260618 22:27:08.884460 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 13708: Not authorized: Client connection negotiation failed: client connection to 127.10.242.193:33887: server requires authentication, but client does not have Kerberos credentials available
I20260618 22:27:08.884642 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 13708
I20260618 22:27:08.897161 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 13842
W20260618 22:27:08.899848 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 13842: Not authorized: Client connection negotiation failed: client connection to 127.10.242.194:36647: server requires authentication, but client does not have Kerberos credentials available
I20260618 22:27:08.900018 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 13842
I20260618 22:27:08.911751 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 13975
W20260618 22:27:08.914686 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 13975: Not authorized: Client connection negotiation failed: client connection to 127.10.242.195:40563: server requires authentication, but client does not have Kerberos credentials available
I20260618 22:27:08.914829 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 13975
I20260618 22:27:08.926811 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 13615
I20260618 22:27:08.928104 13677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:09.028501 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 13615
2026-06-18T22:27:09Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3345 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:27:09 dist-test-slave-0956 krb5kdc[14120](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:27:09 dist-test-slave-0956 krb5kdc[14120](info): set up 2 sockets
Jun 18 22:27:09 dist-test-slave-0956 krb5kdc[14120](info): commencing operation
krb5kdc: starting...
W20260618 22:27:10.033270 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.968s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:27:10 dist-test-slave-0956 krb5kdc[14120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:27:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:27:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:27:10.212174 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:35303
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:46633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:35303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:10.522691 14133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:10.523017 14133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:10.523146 14133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:10.530571 14133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:27:10.530702 14133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:10.530781 14133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:10.530854 14133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:27:10.530915 14133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:27:10.540937 14133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:46633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:35303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:35303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:10.542280 14133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:10.543586 14133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:10.554984 14138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:10.554984 14139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:10.555018 14141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:10.555879 14133 server_base.cc:1034] running on GCE node
I20260618 22:27:10.556721 14133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:10.558223 14133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:10.559420 14133 hybrid_clock.cc:648] HybridClock initialized: now 1781821630559374 us; error 39 us; skew 500 ppm
I20260618 22:27:10.559763 14133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:10 dist-test-slave-0956 krb5kdc[14120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821630, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:10.563053 14133 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:27:10.564528 14133 webserver.cc:460] Webserver started at http://127.10.242.254:39161/ using document root <none> and password file <none>
I20260618 22:27:10.565255 14133 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:10.565361 14133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:10.565627 14133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:10.568507 14133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "7d16e7c0bfae46708dc1effa1eab2a06"
format_stamp: "Formatted at 2026-06-18 22:27:10 on dist-test-slave-0956"
server_key: "63b0c7247a4488c2aa5bcb3a972c661d"
server_key_iv: "3a767739fe651feb0cb9ec84712c528c"
server_key_version: "encryptionkey@0"
I20260618 22:27:10.569316 14133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "7d16e7c0bfae46708dc1effa1eab2a06"
format_stamp: "Formatted at 2026-06-18 22:27:10 on dist-test-slave-0956"
server_key: "63b0c7247a4488c2aa5bcb3a972c661d"
server_key_iv: "3a767739fe651feb0cb9ec84712c528c"
server_key_version: "encryptionkey@0"
I20260618 22:27:10.574553 14133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260618 22:27:10.578298 14148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:10.579689 14133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:27:10.579869 14133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "7d16e7c0bfae46708dc1effa1eab2a06"
format_stamp: "Formatted at 2026-06-18 22:27:10 on dist-test-slave-0956"
server_key: "63b0c7247a4488c2aa5bcb3a972c661d"
server_key_iv: "3a767739fe651feb0cb9ec84712c528c"
server_key_version: "encryptionkey@0"
I20260618 22:27:10.580055 14133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:10.598829 14133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:10.602933 14133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:10.603199 14133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:10.619894 14133 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:35303
I20260618 22:27:10.619915 14199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:35303 every 8 connection(s)
I20260618 22:27:10.621250 14133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:27:10.625007 14200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:10.629139 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 14133
I20260618 22:27:10.629372 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:27:10.630090 11211 external_mini_cluster.cc:1428] Setting key 499aed0e506ea2e88071e110bd064c37
I20260618 22:27:10.636313 14200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06: Bootstrap starting.
I20260618 22:27:10.639415 14200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:10.640386 14200 log.cc:824] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:10.643625 14200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06: No bootstrap required, opened a new log
Jun 18 22:27:10 dist-test-slave-0956 krb5kdc[14120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:10.652216 14200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d16e7c0bfae46708dc1effa1eab2a06" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35303 } }
I20260618 22:27:10.652508 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:10.632023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32900 (local address 127.10.242.254:35303)
0618 22:27:10.632797 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:10.632809 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:10.632849 (+ 40us) server_negotiation.cc:407] Connection header received
0618 22:27:10.635129 (+ 2280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:10.635157 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:10.635602 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:10.636014 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:10.637548 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:10.638850 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:10.640115 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:10.640906 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:10.641146 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:10.645500 (+ 4354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:10.645508 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:10.645523 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:10.645566 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:10.648459 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:10.649302 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:10.649309 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:10.649315 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:10.649493 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:10.649840 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:10.649846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:10.649848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:10.650424 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:10.650643 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:10.651108 (+ 465us) server_negotiation.cc:299] Negotiation successful
0618 22:27:10.651500 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":341,"threads_started":1}
I20260618 22:27:10.652559 14200 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:10.652649 14200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d16e7c0bfae46708dc1effa1eab2a06, State: Initialized, Role: FOLLOWER
I20260618 22:27:10.653285 14200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d16e7c0bfae46708dc1effa1eab2a06" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35303 } }
I20260618 22:27:10.653508 14200 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:27:10.653631 14200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:27:10.653797 14200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:10.656582 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:27:10.656750 14200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d16e7c0bfae46708dc1effa1eab2a06" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35303 } }
I20260618 22:27:10.657341 14200 leader_election.cc:302] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d16e7c0bfae46708dc1effa1eab2a06; no voters:
I20260618 22:27:10.657938 14200 leader_election.cc:288] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:27:10.658073 14205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:10.660089 14200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:27:10.661515 14205 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06 [term 1 LEADER]: Becoming Leader. State: Replica: 7d16e7c0bfae46708dc1effa1eab2a06, State: Running, Role: LEADER
I20260618 22:27:10.662066 14205 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d16e7c0bfae46708dc1effa1eab2a06" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35303 } }
I20260618 22:27:10.664463 14207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d16e7c0bfae46708dc1effa1eab2a06. Latest consensus state: current_term: 1 leader_uuid: "7d16e7c0bfae46708dc1effa1eab2a06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d16e7c0bfae46708dc1effa1eab2a06" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35303 } } }
I20260618 22:27:10.664858 14207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:10.665398 14206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d16e7c0bfae46708dc1effa1eab2a06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d16e7c0bfae46708dc1effa1eab2a06" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35303 } } }
I20260618 22:27:10.665692 14206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:10.669962 14212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:27:10.675364 14212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:27:10.691491 14212 catalog_manager.cc:1347] Generated new cluster ID: eff6d2328de24146a7ec22834d0cd1a2
I20260618 22:27:10.691627 14212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:27:10.701400 14212 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:27:10.703233 14212 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:27:10.717996 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:10.708847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32902 (local address 127.10.242.254:35303)
0618 22:27:10.709041 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:10.709047 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:10.709642 (+ 595us) server_negotiation.cc:407] Connection header received
0618 22:27:10.709822 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:10.709828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:10.709905 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:10.710026 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:10.711261 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:10.712357 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:10.714310 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:10.714939 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:10.715090 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:10.716296 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:10.716302 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:10.716307 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:10.717177 (+ 870us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:10.717252 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:10.717285 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:10.717381 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:10.717539 (+ 158us) server_negotiation.cc:299] Negotiation successful
0618 22:27:10.717637 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260618 22:27:10.723138 14212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d16e7c0bfae46708dc1effa1eab2a06: Generated new TSK 0
I20260618 22:27:10.724321 14212 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:27:10.731580 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 14133
I20260618 22:27:10.732760 14195 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:10.821102 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 14133
2026-06-18T22:27:10Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1789 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:27:10 dist-test-slave-0956 krb5kdc[14234](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:27:10 dist-test-slave-0956 krb5kdc[14234](info): set up 2 sockets
Jun 18 22:27:10 dist-test-slave-0956 krb5kdc[14234](info): commencing operation
krb5kdc: starting...
W20260618 22:27:11.821159 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:27:11 dist-test-slave-0956 krb5kdc[14234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:27:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:27:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:27:12.003930 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:33217
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:33217
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:12.316785 14247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:12.317123 14247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:12.317251 14247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:12.324505 14247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:27:12.324615 14247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:12.324683 14247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:12.324759 14247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:27:12.324813 14247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:27:12.335014 14247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:33217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:33217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:12.336310 14247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:12.337607 14247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:12.348858 14255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:12.348769 14253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:12.348954 14252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:12.350173 14247 server_base.cc:1034] running on GCE node
I20260618 22:27:12.350926 14247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:12.352336 14247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:12.353531 14247 hybrid_clock.cc:648] HybridClock initialized: now 1781821632353483 us; error 38 us; skew 500 ppm
I20260618 22:27:12.353870 14247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:12 dist-test-slave-0956 krb5kdc[14234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821632, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:12.356774 14247 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:27:12.358268 14247 webserver.cc:460] Webserver started at http://127.10.242.254:39077/ using document root <none> and password file <none>
I20260618 22:27:12.358906 14247 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:12.359019 14247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:12.359308 14247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:12.362296 14247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "f8c4c3133f53440a82227bccd1602f88"
format_stamp: "Formatted at 2026-06-18 22:27:12 on dist-test-slave-0956"
server_key: "c5c5620e36bef537dd47ff4a17e62619"
server_key_iv: "7d4fd426fcac2296ba0b3c5eeb7945f3"
server_key_version: "encryptionkey@0"
I20260618 22:27:12.363034 14247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "f8c4c3133f53440a82227bccd1602f88"
format_stamp: "Formatted at 2026-06-18 22:27:12 on dist-test-slave-0956"
server_key: "c5c5620e36bef537dd47ff4a17e62619"
server_key_iv: "7d4fd426fcac2296ba0b3c5eeb7945f3"
server_key_version: "encryptionkey@0"
I20260618 22:27:12.367976 14247 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:27:12.371315 14262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:12.372567 14247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:27:12.372710 14247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "f8c4c3133f53440a82227bccd1602f88"
format_stamp: "Formatted at 2026-06-18 22:27:12 on dist-test-slave-0956"
server_key: "c5c5620e36bef537dd47ff4a17e62619"
server_key_iv: "7d4fd426fcac2296ba0b3c5eeb7945f3"
server_key_version: "encryptionkey@0"
I20260618 22:27:12.372884 14247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:12.391536 14247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:12.395356 14247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:12.395618 14247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:12.412576 14247 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:33217
I20260618 22:27:12.412604 14313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:33217 every 8 connection(s)
I20260618 22:27:12.413954 14247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:27:12.417541 14314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:12.421057 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 14247
I20260618 22:27:12.421300 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:27:12.421849 11211 external_mini_cluster.cc:1428] Setting key efef48241c94df1df76dd5603dcc0c33
I20260618 22:27:12.427915 14314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88: Bootstrap starting.
I20260618 22:27:12.431442 14314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:12.432575 14314 log.cc:824] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:12.435349 14314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88: No bootstrap required, opened a new log
Jun 18 22:27:12 dist-test-slave-0956 krb5kdc[14234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:12.445647 14317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:12.423724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48016 (local address 127.10.242.254:33217)
0618 22:27:12.424442 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:12.424457 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:12.424502 (+ 45us) server_negotiation.cc:407] Connection header received
0618 22:27:12.426466 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:12.426490 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:12.426882 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:12.427209 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:12.428876 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:12.430368 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:12.432064 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:12.432801 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:12.432939 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:12.436965 (+ 4026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:12.436981 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:12.437007 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:12.437052 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:12.441054 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:12.441756 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:12.441769 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:12.441780 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:12.441988 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:12.442300 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:12.442305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:12.442309 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:12.443079 (+ 770us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:12.443380 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:12.443710 (+ 330us) server_negotiation.cc:299] Negotiation successful
0618 22:27:12.444195 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":340,"threads_started":1}
I20260618 22:27:12.445648 14314 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8c4c3133f53440a82227bccd1602f88" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33217 } }
I20260618 22:27:12.446178 14314 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:12.446298 14314 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8c4c3133f53440a82227bccd1602f88, State: Initialized, Role: FOLLOWER
I20260618 22:27:12.446955 14314 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8c4c3133f53440a82227bccd1602f88" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33217 } }
I20260618 22:27:12.447223 14314 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:27:12.447356 14314 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:27:12.447515 14314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:12.449950 14314 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8c4c3133f53440a82227bccd1602f88" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33217 } }
I20260618 22:27:12.450474 14314 leader_election.cc:302] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8c4c3133f53440a82227bccd1602f88; no voters:
I20260618 22:27:12.451071 14314 leader_election.cc:288] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:27:12.451164 14319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:12.451802 14319 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88 [term 1 LEADER]: Becoming Leader. State: Replica: f8c4c3133f53440a82227bccd1602f88, State: Running, Role: LEADER
I20260618 22:27:12.452281 14319 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8c4c3133f53440a82227bccd1602f88" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33217 } }
I20260618 22:27:12.453042 14314 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:27:12.455092 14320 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8c4c3133f53440a82227bccd1602f88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8c4c3133f53440a82227bccd1602f88" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33217 } } }
I20260618 22:27:12.455550 14320 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:12.455767 14321 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8c4c3133f53440a82227bccd1602f88. Latest consensus state: current_term: 1 leader_uuid: "f8c4c3133f53440a82227bccd1602f88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8c4c3133f53440a82227bccd1602f88" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33217 } } }
I20260618 22:27:12.456161 14321 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:12.458388 14328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:27:12.463898 14328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:27:12.472872 14328 catalog_manager.cc:1347] Generated new cluster ID: 7237d926cd694a64a58a168e31c2dd0f
I20260618 22:27:12.473002 14328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:27:12.487082 14328 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:27:12.489140 14328 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:27:12.501264 14328 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8c4c3133f53440a82227bccd1602f88: Generated new TSK 0
I20260618 22:27:12.502522 14328 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
I20260618 22:27:12.548744 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:33217
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:12.866631 14340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:12.866930 14340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:12.867067 14340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:12.874290 14340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:12.874477 14340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:12.874646 14340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:27:12.884459 14340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:33217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:12.885852 14340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:12.887113 14340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:12.899611 14345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:12.899595 14346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:12.899680 14348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:12.900274 14340 server_base.cc:1034] running on GCE node
I20260618 22:27:12.900885 14340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:12.901885 14340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:12.903069 14340 hybrid_clock.cc:648] HybridClock initialized: now 1781821632903009 us; error 56 us; skew 500 ppm
I20260618 22:27:12.903409 14340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:12 dist-test-slave-0956 krb5kdc[14234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821632, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:12.906612 14340 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:27:12.908218 14340 webserver.cc:460] Webserver started at http://127.10.242.193:34589/ using document root <none> and password file <none>
I20260618 22:27:12.908890 14340 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:12.908991 14340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:12.909294 14340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:12.912086 14340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "0bab731efd9d40078c6c6cd08d6f18cf"
format_stamp: "Formatted at 2026-06-18 22:27:12 on dist-test-slave-0956"
server_key: "19951500356dc2436a7081b26266aaf3"
server_key_iv: "b87a8d8b0a7d0c79ffe09ab34a1c1f62"
server_key_version: "encryptionkey@0"
I20260618 22:27:12.912887 14340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "0bab731efd9d40078c6c6cd08d6f18cf"
format_stamp: "Formatted at 2026-06-18 22:27:12 on dist-test-slave-0956"
server_key: "19951500356dc2436a7081b26266aaf3"
server_key_iv: "b87a8d8b0a7d0c79ffe09ab34a1c1f62"
server_key_version: "encryptionkey@0"
I20260618 22:27:12.918037 14340 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260618 22:27:12.921465 14355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:12.922887 14340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:27:12.923095 14340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "0bab731efd9d40078c6c6cd08d6f18cf"
format_stamp: "Formatted at 2026-06-18 22:27:12 on dist-test-slave-0956"
server_key: "19951500356dc2436a7081b26266aaf3"
server_key_iv: "b87a8d8b0a7d0c79ffe09ab34a1c1f62"
server_key_version: "encryptionkey@0"
I20260618 22:27:12.923290 14340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:12.940307 14340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:12.944725 14340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:12.944993 14340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:12.945964 14340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:12.947258 14340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:12.947332 14340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:12.947445 14340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:12.947503 14340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:12.978251 14340 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:45503
I20260618 22:27:12.978283 14467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:45503 every 8 connection(s)
I20260618 22:27:12.979462 14340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:27:12.988124 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 14340
I20260618 22:27:12.988349 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:27:12.989132 11211 external_mini_cluster.cc:1428] Setting key 33bf3f2a1f47e869405aab98484c80d9
Jun 18 22:27:12 dist-test-slave-0956 krb5kdc[14234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821632, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:13.000108 14470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:12.981818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:46913 (local address 127.10.242.254:33217)
0618 22:27:12.982190 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:12.982195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:12.983159 (+ 964us) server_negotiation.cc:407] Connection header received
0618 22:27:12.984877 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:12.984882 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:12.984965 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:12.985060 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:12.986601 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:12.987299 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:12.989488 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:12.990011 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:12.990113 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:12.994130 (+ 4017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:12.994138 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:12.994143 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:12.994187 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:12.996969 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:12.997853 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:12.997857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:12.997860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:12.997980 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:12.998428 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:12.998431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:12.998433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:12.998708 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:12.998813 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:12.999666 (+ 853us) server_negotiation.cc:299] Negotiation successful
0618 22:27:12.999827 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":176,"threads_started":1}
I20260618 22:27:13.001451 14471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:12.982337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:33217 (local address 127.10.242.193:46913)
0618 22:27:12.983000 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:12.983037 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:12.984616 (+ 1579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:12.985344 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:12.985352 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:12.985878 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:12.986439 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:12.986458 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:12.987443 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:12.987447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:12.989347 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:12.989362 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:12.990255 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:12.990261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:12.990458 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:12.991126 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:12.991155 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:12.993815 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:12.997175 (+ 3360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:12.997183 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:12.997198 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:12.997721 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:12.998070 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:12.998075 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:12.998079 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:12.998314 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:12.998815 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:12.998823 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:12.999261 (+ 438us) client_negotiation.cc:769] Sending connection context
0618 22:27:12.999573 (+ 312us) client_negotiation.cc:240] Negotiation successful
0618 22:27:13.000090 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":256,"threads_started":1}
I20260618 22:27:13.002908 14468 heartbeater.cc:347] Connected to a master server at 127.10.242.254:33217
I20260618 22:27:13.003217 14468 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:13.004118 14468 heartbeater.cc:511] Master 127.10.242.254:33217 requested a full tablet report, sending...
I20260618 22:27:13.006256 14279 ts_manager.cc:194] Registered new tserver with Master: 0bab731efd9d40078c6c6cd08d6f18cf (127.10.242.193:45503)
WARNING: no policy specified for kudu/127.10.242.194@KRBTEST.COM; defaulting to no policy
I20260618 22:27:13.007718 14279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:46913
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
I20260618 22:27:13.030565 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:33217
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:13.338814 14474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:13.339085 14474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:13.339175 14474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:13.346371 14474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:13.346479 14474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:13.346597 14474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:27:13.356189 14474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:33217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:13.357486 14474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:13.358830 14474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:13.370957 14479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:13.371207 14482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:13.371358 14480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:13.372220 14474 server_base.cc:1034] running on GCE node
I20260618 22:27:13.372788 14474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:13.373729 14474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:13.374929 14474 hybrid_clock.cc:648] HybridClock initialized: now 1781821633374878 us; error 48 us; skew 500 ppm
I20260618 22:27:13.375227 14474 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:13 dist-test-slave-0956 krb5kdc[14234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821633, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:13.382507 14474 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:27:13.383994 14474 webserver.cc:460] Webserver started at http://127.10.242.194:35263/ using document root <none> and password file <none>
I20260618 22:27:13.384665 14474 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:13.384792 14474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:13.385068 14474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:13.387902 14474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "2647d4080de7426aa3e63cc2e84c97c6"
format_stamp: "Formatted at 2026-06-18 22:27:13 on dist-test-slave-0956"
server_key: "02db63469c76f830ac69871ff26e390c"
server_key_iv: "06f3b131daac8bb0f214068e6ed9b34f"
server_key_version: "encryptionkey@0"
I20260618 22:27:13.388616 14474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "2647d4080de7426aa3e63cc2e84c97c6"
format_stamp: "Formatted at 2026-06-18 22:27:13 on dist-test-slave-0956"
server_key: "02db63469c76f830ac69871ff26e390c"
server_key_iv: "06f3b131daac8bb0f214068e6ed9b34f"
server_key_version: "encryptionkey@0"
I20260618 22:27:13.393548 14474 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260618 22:27:13.396865 14489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:13.398221 14474 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:27:13.398368 14474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "2647d4080de7426aa3e63cc2e84c97c6"
format_stamp: "Formatted at 2026-06-18 22:27:13 on dist-test-slave-0956"
server_key: "02db63469c76f830ac69871ff26e390c"
server_key_iv: "06f3b131daac8bb0f214068e6ed9b34f"
server_key_version: "encryptionkey@0"
I20260618 22:27:13.398537 14474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:13.415555 14474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:13.419588 14474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:13.419852 14474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:13.420846 14474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:13.422287 14474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:13.422361 14474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:13.422482 14474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:13.422533 14474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:13.452142 14474 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:34131
I20260618 22:27:13.452176 14601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:34131 every 8 connection(s)
I20260618 22:27:13.453454 14474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:27:13.457252 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 14474
I20260618 22:27:13.457428 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:27:13.458016 11211 external_mini_cluster.cc:1428] Setting key 28f1496cb65cd21a8643ad35d8441326
Jun 18 22:27:13 dist-test-slave-0956 krb5kdc[14234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821633, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:13.474615 14470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:13.455889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:51703 (local address 127.10.242.254:33217)
0618 22:27:13.456043 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:13.456047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:13.457265 (+ 1218us) server_negotiation.cc:407] Connection header received
0618 22:27:13.459427 (+ 2162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:13.459432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:13.459511 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:13.459599 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:13.461447 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:13.462152 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:13.464158 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:13.464534 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:13.464636 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:13.468314 (+ 3678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:13.468321 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:13.468326 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:13.468363 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:13.471446 (+ 3083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:13.472206 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:13.472211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:13.472215 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:13.472370 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:13.472781 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:13.472787 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:13.472790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:13.473179 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:13.473329 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:13.474044 (+ 715us) server_negotiation.cc:299] Negotiation successful
0618 22:27:13.474280 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260618 22:27:13.475457 14604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:13.456334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:33217 (local address 127.10.242.194:51703)
0618 22:27:13.457064 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:13.457142 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:13.459124 (+ 1982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:13.459866 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:13.459877 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:13.460485 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:13.461277 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:13.461302 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:13.462289 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:13.462294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:13.464027 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:13.464042 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:13.464728 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:13.464734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:13.464996 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:13.465746 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:13.465771 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:13.468062 (+ 2291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:13.471597 (+ 3535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:13.471605 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:13.471619 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:13.472054 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:13.472467 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:13.472470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:13.472472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:13.472664 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:13.473312 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:13.473318 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:13.473671 (+ 353us) client_negotiation.cc:769] Sending connection context
0618 22:27:13.473905 (+ 234us) client_negotiation.cc:240] Negotiation successful
0618 22:27:13.474307 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":253,"threads_started":1}
WARNING: no policy specified for kudu/127.10.242.195@KRBTEST.COM; defaulting to no policy
I20260618 22:27:13.476868 14602 heartbeater.cc:347] Connected to a master server at 127.10.242.254:33217
I20260618 22:27:13.477223 14602 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:13.477979 14602 heartbeater.cc:511] Master 127.10.242.254:33217 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.195@KRBTEST.COM" created.
I20260618 22:27:13.479486 14279 ts_manager.cc:194] Registered new tserver with Master: 2647d4080de7426aa3e63cc2e84c97c6 (127.10.242.194:34131)
I20260618 22:27:13.480275 14279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:51703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
I20260618 22:27:13.500394 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:33217
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:13.815480 14607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:13.815768 14607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:13.815888 14607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:13.823037 14607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:13.823171 14607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:13.823316 14607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:27:13.833354 14607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:33217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:13.834667 14607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:13.835914 14607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:13.848489 14615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:13.848531 14612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:13.848681 14613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:13.849071 14607 server_base.cc:1034] running on GCE node
I20260618 22:27:13.849687 14607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:13.850636 14607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:13.851853 14607 hybrid_clock.cc:648] HybridClock initialized: now 1781821633851792 us; error 54 us; skew 500 ppm
I20260618 22:27:13.852216 14607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:13 dist-test-slave-0956 krb5kdc[14234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821633, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:13.855466 14607 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:27:13.856989 14607 webserver.cc:460] Webserver started at http://127.10.242.195:43903/ using document root <none> and password file <none>
I20260618 22:27:13.857697 14607 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:13.857811 14607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:13.858104 14607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:13.860985 14607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "5f151fa294c14de38e6d46f50766bbfb"
format_stamp: "Formatted at 2026-06-18 22:27:13 on dist-test-slave-0956"
server_key: "b8d8a617d72c87293c695524c19c06d3"
server_key_iv: "f3cfe7abd01cdcefcd5e4e63428b8491"
server_key_version: "encryptionkey@0"
I20260618 22:27:13.861764 14607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "5f151fa294c14de38e6d46f50766bbfb"
format_stamp: "Formatted at 2026-06-18 22:27:13 on dist-test-slave-0956"
server_key: "b8d8a617d72c87293c695524c19c06d3"
server_key_iv: "f3cfe7abd01cdcefcd5e4e63428b8491"
server_key_version: "encryptionkey@0"
I20260618 22:27:13.866900 14607 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:27:13.870297 14622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:13.871621 14607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:27:13.871770 14607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "5f151fa294c14de38e6d46f50766bbfb"
format_stamp: "Formatted at 2026-06-18 22:27:13 on dist-test-slave-0956"
server_key: "b8d8a617d72c87293c695524c19c06d3"
server_key_iv: "f3cfe7abd01cdcefcd5e4e63428b8491"
server_key_version: "encryptionkey@0"
I20260618 22:27:13.871951 14607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:13.885667 14607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:13.889669 14607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:13.889941 14607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:13.890834 14607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:13.892176 14607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:13.892253 14607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:13.892369 14607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:13.892419 14607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:13.921334 14607 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:39331
I20260618 22:27:13.921367 14734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:39331 every 8 connection(s)
I20260618 22:27:13.922605 14607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:27:13.927522 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 14607
I20260618 22:27:13.927700 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:27:13.928167 11211 external_mini_cluster.cc:1428] Setting key 92f28c3dfd06ad0316437f0eebb62cf9
Jun 18 22:27:13 dist-test-slave-0956 krb5kdc[14234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821633, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:13.941547 14470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:13.925247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:58447 (local address 127.10.242.254:33217)
0618 22:27:13.925423 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:13.925427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:13.926555 (+ 1128us) server_negotiation.cc:407] Connection header received
0618 22:27:13.928445 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:13.928450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:13.928543 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:13.928654 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:13.930515 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:13.931455 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:13.933046 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:13.933495 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:13.933585 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:13.936712 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:13.936717 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:13.936720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:13.936752 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:13.939019 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:13.939642 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:13.939646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:13.939648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:13.939752 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:13.940071 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:13.940074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:13.940076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:13.940334 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:13.940435 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:13.941068 (+ 633us) server_negotiation.cc:299] Negotiation successful
0618 22:27:13.941266 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260618 22:27:13.942590 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:13.925718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:33217 (local address 127.10.242.195:58447)
0618 22:27:13.926404 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:13.926441 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:13.928134 (+ 1693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:13.928937 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:13.928950 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:13.929639 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:13.930331 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:13.930358 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:13.931583 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:13.931587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:13.932921 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:13.932933 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:13.933683 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:13.933688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:13.933878 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:13.934451 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:13.934473 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:13.936502 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:13.939133 (+ 2631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:13.939140 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:13.939153 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:13.939543 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:13.939825 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:13.939828 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:13.939830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:13.939999 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:13.940419 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:13.940425 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:13.940753 (+ 328us) client_negotiation.cc:769] Sending connection context
0618 22:27:13.940987 (+ 234us) client_negotiation.cc:240] Negotiation successful
0618 22:27:13.941427 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":256,"threads_started":1}
I20260618 22:27:13.943943 14735 heartbeater.cc:347] Connected to a master server at 127.10.242.254:33217
I20260618 22:27:13.944301 14735 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:13.945031 14735 heartbeater.cc:511] Master 127.10.242.254:33217 requested a full tablet report, sending...
I20260618 22:27:13.946484 14279 ts_manager.cc:194] Registered new tserver with Master: 5f151fa294c14de38e6d46f50766bbfb (127.10.242.195:39331)
I20260618 22:27:13.947258 14279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:58447
I20260618 22:27:13.955649 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 18 22:27:13 dist-test-slave-0956 krb5kdc[14234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821633, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 18 22:27:13 dist-test-slave-0956 krb5kdc[14234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821633, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:14.002429 14470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:13.988901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48026 (local address 127.10.242.254:33217)
0618 22:27:13.989171 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:13.989179 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:13.989591 (+ 412us) server_negotiation.cc:407] Connection header received
0618 22:27:13.989732 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:13.989736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:13.989800 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:13.989878 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:13.991132 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:13.991835 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:13.993019 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:13.993471 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:13.993566 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:13.997240 (+ 3674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:13.997244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:13.997247 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:13.997283 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:13.999784 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:14.000458 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:14.000462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:14.000465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:14.000599 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:14.000962 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:14.000966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:14.000968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:14.001366 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:14.001587 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:14.001886 (+ 299us) server_negotiation.cc:299] Negotiation successful
0618 22:27:14.002120 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260618 22:27:14.002938 14279 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48026
I20260618 22:27:14.004359 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 14340
I20260618 22:27:14.010913 14468 heartbeater.cc:503] Master 127.10.242.254:33217 was elected leader, sending a full tablet report...
Jun 18 22:27:14 dist-test-slave-0956 krb5kdc[14234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821633, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:27:14.019745 14746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:14.005364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40332 (local address 127.10.242.193:45503)
0618 22:27:14.005938 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:14.005944 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:14.005969 (+ 25us) server_negotiation.cc:407] Connection header received
0618 22:27:14.006020 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:14.006026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:14.006246 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:14.006370 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:14.007633 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:14.008688 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:14.009934 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:14.010532 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:14.010629 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:14.014685 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:14.014698 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:14.014714 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:14.014757 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:14.017519 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:14.018105 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:14.018112 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:14.018114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:14.018232 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:14.018496 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:14.018499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:14.018501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:14.018823 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:14.019045 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:14.019211 (+ 166us) server_negotiation.cc:299] Negotiation successful
0618 22:27:14.019397 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":301,"threads_started":1}
W20260618 22:27:14.020991 14463 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40332
W20260618 22:27:14.022270 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 14340: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:27:14.022429 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 14340
I20260618 22:27:14.034281 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 14474
Jun 18 22:27:14 dist-test-slave-0956 krb5kdc[14234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821633, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.242.194@KRBTEST.COM
I20260618 22:27:14.048942 14747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:14.035069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37986 (local address 127.10.242.194:34131)
0618 22:27:14.035612 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:14.035619 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:14.035644 (+ 25us) server_negotiation.cc:407] Connection header received
0618 22:27:14.035709 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:14.035716 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:14.035922 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:14.036052 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:14.037197 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:14.038181 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:14.039337 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:14.039862 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:14.039953 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:14.043891 (+ 3938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:14.043903 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:14.043917 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:14.043950 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:14.046600 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:14.047199 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:14.047206 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:14.047208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:14.047338 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:14.047638 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:14.047641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:14.047643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:14.047947 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:14.048168 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:14.048462 (+ 294us) server_negotiation.cc:299] Negotiation successful
0618 22:27:14.048644 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":289,"threads_started":1}
W20260618 22:27:14.050158 14597 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37986
W20260618 22:27:14.051199 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 14474: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:27:14.051345 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 14474
I20260618 22:27:14.062965 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 14607
Jun 18 22:27:14 dist-test-slave-0956 krb5kdc[14234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821633, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.242.195@KRBTEST.COM
I20260618 22:27:14.078471 14748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:14.063881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44730 (local address 127.10.242.195:39331)
0618 22:27:14.064485 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:14.064491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:14.064522 (+ 31us) server_negotiation.cc:407] Connection header received
0618 22:27:14.064578 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:14.064584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:14.064785 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:14.064930 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:14.066222 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:14.067244 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:14.068433 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:14.068980 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:14.069135 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:14.073198 (+ 4063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:14.073210 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:14.073226 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:14.073259 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:14.075796 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:14.076454 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:14.076462 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:14.076464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:14.076609 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:14.076930 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:14.076934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:14.076938 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:14.077323 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:14.077621 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:14.077883 (+ 262us) server_negotiation.cc:299] Negotiation successful
0618 22:27:14.078134 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":325,"threads_started":1}
W20260618 22:27:14.079695 14730 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44730
W20260618 22:27:14.080801 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 14607: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:27:14.080935 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 14607
I20260618 22:27:14.092446 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 14247
I20260618 22:27:14.093626 14309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:14.192858 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 14247
2026-06-18T22:27:14Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3374 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:27:14 dist-test-slave-0956 krb5kdc[14752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:27:14 dist-test-slave-0956 krb5kdc[14752](info): set up 2 sockets
Jun 18 22:27:14 dist-test-slave-0956 krb5kdc[14752](info): commencing operation
krb5kdc: starting...
W20260618 22:27:15.177230 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.003s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:27:15 dist-test-slave-0956 krb5kdc[14752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:27:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:27:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:27:15.357981 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:44713
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:41451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:44713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:15.666641 14765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:15.666946 14765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:15.667091 14765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:15.674268 14765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:27:15.674393 14765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:15.674471 14765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:15.674574 14765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:27:15.674635 14765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:27:15.684407 14765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:41451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:44713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:44713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:15.685794 14765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:15.687075 14765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:15.698176 14770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:15.698196 14773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:15.698253 14771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:15.698975 14765 server_base.cc:1034] running on GCE node
I20260618 22:27:15.699807 14765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:15.701345 14765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:15.702610 14765 hybrid_clock.cc:648] HybridClock initialized: now 1781821635702565 us; error 38 us; skew 500 ppm
I20260618 22:27:15.702986 14765 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:15 dist-test-slave-0956 krb5kdc[14752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821635, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:15.706269 14765 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:27:15.707772 14765 webserver.cc:460] Webserver started at http://127.10.242.254:35573/ using document root <none> and password file <none>
I20260618 22:27:15.708442 14765 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:15.708551 14765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:15.708865 14765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:15.712132 14765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "9404718e4e124fceb929f005062acac9"
format_stamp: "Formatted at 2026-06-18 22:27:15 on dist-test-slave-0956"
server_key: "33769c17f7096de80ba703ce5c77acce"
server_key_iv: "e0bcd6033c0f71d957303b94bfb4e6d7"
server_key_version: "encryptionkey@0"
I20260618 22:27:15.712924 14765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "9404718e4e124fceb929f005062acac9"
format_stamp: "Formatted at 2026-06-18 22:27:15 on dist-test-slave-0956"
server_key: "33769c17f7096de80ba703ce5c77acce"
server_key_iv: "e0bcd6033c0f71d957303b94bfb4e6d7"
server_key_version: "encryptionkey@0"
I20260618 22:27:15.718097 14765 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260618 22:27:15.721611 14780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:15.723023 14765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260618 22:27:15.723201 14765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "9404718e4e124fceb929f005062acac9"
format_stamp: "Formatted at 2026-06-18 22:27:15 on dist-test-slave-0956"
server_key: "33769c17f7096de80ba703ce5c77acce"
server_key_iv: "e0bcd6033c0f71d957303b94bfb4e6d7"
server_key_version: "encryptionkey@0"
I20260618 22:27:15.723390 14765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:15.748080 14765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:15.752347 14765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:15.752610 14765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:15.769742 14765 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:44713
I20260618 22:27:15.769770 14831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:44713 every 8 connection(s)
I20260618 22:27:15.771103 14765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:27:15.774959 14832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:15.775305 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 14765
I20260618 22:27:15.775496 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:27:15.776299 11211 external_mini_cluster.cc:1428] Setting key 195cb63ddd2347c2218d29e4765d86e4
I20260618 22:27:15.788980 14832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9: Bootstrap starting.
Jun 18 22:27:15 dist-test-slave-0956 krb5kdc[14752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:15.792758 14832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:15.793882 14832 log.cc:824] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:15.797226 14832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9: No bootstrap required, opened a new log
I20260618 22:27:15.800000 14835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:15.778504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50672 (local address 127.10.242.254:44713)
0618 22:27:15.779274 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:15.779287 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:15.779328 (+ 41us) server_negotiation.cc:407] Connection header received
0618 22:27:15.781290 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:15.781315 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:15.781759 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:15.782118 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:15.783904 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:15.785289 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:15.787014 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:15.787704 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:15.787836 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:15.792313 (+ 4477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:15.792321 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:15.792338 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:15.792380 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:15.795602 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:15.796448 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:15.796455 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:15.796462 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:15.796620 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:15.797148 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:15.797153 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:15.797155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:15.797755 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:15.797998 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:15.798434 (+ 436us) server_negotiation.cc:299] Negotiation successful
0618 22:27:15.798804 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":348,"threads_started":1}
I20260618 22:27:15.805256 14832 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9404718e4e124fceb929f005062acac9" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44713 } }
I20260618 22:27:15.805665 14832 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:15.805763 14832 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9404718e4e124fceb929f005062acac9, State: Initialized, Role: FOLLOWER
I20260618 22:27:15.806308 14832 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9404718e4e124fceb929f005062acac9" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44713 } }
I20260618 22:27:15.806526 14832 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:27:15.806651 14832 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:27:15.806829 14832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:15.809034 14832 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9404718e4e124fceb929f005062acac9" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44713 } }
I20260618 22:27:15.809574 14832 leader_election.cc:302] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9404718e4e124fceb929f005062acac9; no voters:
I20260618 22:27:15.810417 14832 leader_election.cc:288] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:27:15.810528 14837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:15.811255 14837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9 [term 1 LEADER]: Becoming Leader. State: Replica: 9404718e4e124fceb929f005062acac9, State: Running, Role: LEADER
I20260618 22:27:15.812140 14837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9404718e4e124fceb929f005062acac9" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44713 } }
I20260618 22:27:15.813598 14832 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:27:15.815671 14839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9404718e4e124fceb929f005062acac9. Latest consensus state: current_term: 1 leader_uuid: "9404718e4e124fceb929f005062acac9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9404718e4e124fceb929f005062acac9" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44713 } } }
I20260618 22:27:15.816047 14839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:15.816210 14838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9404718e4e124fceb929f005062acac9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9404718e4e124fceb929f005062acac9" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44713 } } }
I20260618 22:27:15.816601 14838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:15.817385 14843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:27:15.822238 14843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:27:15.833585 14843 catalog_manager.cc:1347] Generated new cluster ID: 96930d122d6645c3bea3672c893623f9
I20260618 22:27:15.833712 14843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:27:15.854868 14843 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:27:15.856323 14843 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:27:15.863519 14843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9404718e4e124fceb929f005062acac9: Generated new TSK 0
I20260618 22:27:15.864383 14843 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
I20260618 22:27:15.918555 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:44713
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:41451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:16.245298 14858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:16.245604 14858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:16.245734 14858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:16.253165 14858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:16.253302 14858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:16.253458 14858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:27:16.263228 14858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:41451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:44713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:16.264587 14858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:16.265897 14858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:16.278813 14863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:16.278752 14864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:16.278785 14866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:16.279553 14858 server_base.cc:1034] running on GCE node
I20260618 22:27:16.280131 14858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:16.281039 14858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:16.282323 14858 hybrid_clock.cc:648] HybridClock initialized: now 1781821636282268 us; error 43 us; skew 500 ppm
I20260618 22:27:16.282650 14858 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:16 dist-test-slave-0956 krb5kdc[14752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821636, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:16.285705 14858 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:27:16.287098 14858 webserver.cc:460] Webserver started at http://127.10.242.193:45791/ using document root <none> and password file <none>
I20260618 22:27:16.287727 14858 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:16.287825 14858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:16.288146 14858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:16.290998 14858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "0022c6aa0d914eaba8f4cf49f0e5a9ed"
format_stamp: "Formatted at 2026-06-18 22:27:16 on dist-test-slave-0956"
server_key: "0fc4dbc01d77f9ef33ef40e1ecc24937"
server_key_iv: "1ab2425d6526b1394551eb60c15ffba1"
server_key_version: "encryptionkey@0"
I20260618 22:27:16.291719 14858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "0022c6aa0d914eaba8f4cf49f0e5a9ed"
format_stamp: "Formatted at 2026-06-18 22:27:16 on dist-test-slave-0956"
server_key: "0fc4dbc01d77f9ef33ef40e1ecc24937"
server_key_iv: "1ab2425d6526b1394551eb60c15ffba1"
server_key_version: "encryptionkey@0"
I20260618 22:27:16.296622 14858 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260618 22:27:16.300009 14873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:16.301487 14858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:27:16.301699 14858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "0022c6aa0d914eaba8f4cf49f0e5a9ed"
format_stamp: "Formatted at 2026-06-18 22:27:16 on dist-test-slave-0956"
server_key: "0fc4dbc01d77f9ef33ef40e1ecc24937"
server_key_iv: "1ab2425d6526b1394551eb60c15ffba1"
server_key_version: "encryptionkey@0"
I20260618 22:27:16.301918 14858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:16.315575 14858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:16.319931 14858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:16.320235 14858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:16.321230 14858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:16.322649 14858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:16.322736 14858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:16.322860 14858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:16.322916 14858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:16.354131 14858 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:42421
I20260618 22:27:16.354164 14985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:42421 every 8 connection(s)
I20260618 22:27:16.355466 14858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:27:16.358515 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 14858
I20260618 22:27:16.358709 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:27:16.359308 11211 external_mini_cluster.cc:1428] Setting key 25eef1ea375dd3c519c56acbc6e8631d
Jun 18 22:27:16 dist-test-slave-0956 krb5kdc[14752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821636, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.242.194@KRBTEST.COM; defaulting to no policy
I20260618 22:27:16.377151 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:16.357969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:58733 (local address 127.10.242.254:44713)
0618 22:27:16.358438 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:16.358444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:16.359548 (+ 1104us) server_negotiation.cc:407] Connection header received
0618 22:27:16.361587 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:16.361593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:16.361685 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:16.361798 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:16.363519 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:16.364508 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:16.366306 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:16.366908 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:16.367017 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:16.371019 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:16.371024 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:16.371028 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:16.371067 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:16.373619 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:16.374607 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:16.374611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:16.374613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:16.374735 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:16.375261 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:16.375265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:16.375268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:16.375552 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:16.375669 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:16.376614 (+ 945us) server_negotiation.cc:299] Negotiation successful
0618 22:27:16.376788 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":244,"threads_started":1}
I20260618 22:27:16.378264 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:16.358569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:44713 (local address 127.10.242.193:58733)
0618 22:27:16.359385 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:16.359431 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:16.361327 (+ 1896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:16.361987 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:16.361995 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:16.362647 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:16.363345 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:16.363365 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:16.364669 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:16.364673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:16.366126 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:16.366137 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:16.367134 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:16.367140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:16.367454 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:16.368122 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:16.368150 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:16.370775 (+ 2625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:16.373783 (+ 3008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:16.373793 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:16.373812 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:16.374478 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:16.374835 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:16.374840 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:16.374844 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:16.375105 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:16.375682 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:16.375690 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:16.376143 (+ 453us) client_negotiation.cc:769] Sending connection context
0618 22:27:16.376483 (+ 340us) client_negotiation.cc:240] Negotiation successful
0618 22:27:16.377054 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":338,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.194@KRBTEST.COM" created.
I20260618 22:27:16.379930 14986 heartbeater.cc:347] Connected to a master server at 127.10.242.254:44713
I20260618 22:27:16.380317 14986 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:16.381260 14986 heartbeater.cc:511] Master 127.10.242.254:44713 requested a full tablet report, sending...
I20260618 22:27:16.383550 14797 ts_manager.cc:194] Registered new tserver with Master: 0022c6aa0d914eaba8f4cf49f0e5a9ed (127.10.242.193:42421)
I20260618 22:27:16.384972 14797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:58733
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
I20260618 22:27:16.402472 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:44713
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:41451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:16.713514 14992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:16.713824 14992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:16.713948 14992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:16.721539 14992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:16.721650 14992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:16.721799 14992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:27:16.731760 14992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:41451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:44713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:16.733131 14992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:16.734520 14992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:16.747732 14997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:16.747648 14998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:16.747695 15000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:16.748368 14992 server_base.cc:1034] running on GCE node
I20260618 22:27:16.748957 14992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:16.750020 14992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:16.751255 14992 hybrid_clock.cc:648] HybridClock initialized: now 1781821636751179 us; error 64 us; skew 500 ppm
I20260618 22:27:16.751550 14992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:16 dist-test-slave-0956 krb5kdc[14752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821636, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:16.754770 14992 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:27:16.756271 14992 webserver.cc:460] Webserver started at http://127.10.242.194:46523/ using document root <none> and password file <none>
I20260618 22:27:16.756918 14992 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:16.757026 14992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:16.757485 14992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:16.760692 14992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "ba2fc6689640424796cc08c15ad91b7d"
format_stamp: "Formatted at 2026-06-18 22:27:16 on dist-test-slave-0956"
server_key: "d16ec9c4884b1fef66552325aa4acf9d"
server_key_iv: "48a68ce23fa228880158d4b58b5808f3"
server_key_version: "encryptionkey@0"
I20260618 22:27:16.761533 14992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "ba2fc6689640424796cc08c15ad91b7d"
format_stamp: "Formatted at 2026-06-18 22:27:16 on dist-test-slave-0956"
server_key: "d16ec9c4884b1fef66552325aa4acf9d"
server_key_iv: "48a68ce23fa228880158d4b58b5808f3"
server_key_version: "encryptionkey@0"
I20260618 22:27:16.766734 14992 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:27:16.770401 15007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:16.771854 14992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:27:16.772018 14992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "ba2fc6689640424796cc08c15ad91b7d"
format_stamp: "Formatted at 2026-06-18 22:27:16 on dist-test-slave-0956"
server_key: "d16ec9c4884b1fef66552325aa4acf9d"
server_key_iv: "48a68ce23fa228880158d4b58b5808f3"
server_key_version: "encryptionkey@0"
I20260618 22:27:16.772203 14992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:16.803818 14992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:16.808393 14992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:16.808655 14992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:16.809702 14992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:16.810977 14992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:16.811053 14992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:16.811170 14992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:16.811235 14992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:16.840903 14992 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:43899
I20260618 22:27:16.840938 15119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:43899 every 8 connection(s)
I20260618 22:27:16.842242 14992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:27:16.849905 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 14992
I20260618 22:27:16.850068 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:27:16.850594 11211 external_mini_cluster.cc:1428] Setting key fb44e3eea26135c54c7f090f8060e5b7
Jun 18 22:27:16 dist-test-slave-0956 krb5kdc[14752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821636, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:16.865878 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:16.844762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:51381 (local address 127.10.242.254:44713)
0618 22:27:16.844955 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:16.844961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:16.848500 (+ 3539us) server_negotiation.cc:407] Connection header received
0618 22:27:16.851116 (+ 2616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:16.851122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:16.851220 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:16.851334 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:16.853483 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:16.854204 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:16.855759 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:16.856142 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:16.856252 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:16.860034 (+ 3782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:16.860039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:16.860043 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:16.860085 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:16.862514 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:16.863458 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:16.863463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:16.863465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:16.863645 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:16.864134 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:16.864139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:16.864143 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:16.864424 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:16.864553 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:16.865314 (+ 761us) server_negotiation.cc:299] Negotiation successful
0618 22:27:16.865536 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:27:16.866909 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:16.845442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:44713 (local address 127.10.242.194:51381)
0618 22:27:16.848287 (+ 2845us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:16.848350 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:16.850724 (+ 2374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:16.851678 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:16.851692 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:16.852494 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:16.853291 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:16.853320 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:16.854363 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:16.854368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:16.855621 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:16.855633 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:16.856358 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:16.856365 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:16.856601 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:16.857303 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:16.857327 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:16.859761 (+ 2434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:16.862684 (+ 2923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:16.862695 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:16.862714 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:16.863314 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:16.863793 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:16.863798 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:16.863802 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:16.864015 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:16.864546 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:16.864553 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:16.864921 (+ 368us) client_negotiation.cc:769] Sending connection context
0618 22:27:16.865211 (+ 290us) client_negotiation.cc:240] Negotiation successful
0618 22:27:16.865770 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2513,"thread_start_us":2317,"threads_started":1}
WARNING: no policy specified for kudu/127.10.242.195@KRBTEST.COM; defaulting to no policy
I20260618 22:27:16.868335 15120 heartbeater.cc:347] Connected to a master server at 127.10.242.254:44713
I20260618 22:27:16.868615 15120 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:16.869423 15120 heartbeater.cc:511] Master 127.10.242.254:44713 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.195@KRBTEST.COM" created.
I20260618 22:27:16.870929 14797 ts_manager.cc:194] Registered new tserver with Master: ba2fc6689640424796cc08c15ad91b7d (127.10.242.194:43899)
I20260618 22:27:16.871716 14797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:51381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
I20260618 22:27:16.892994 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:44713
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:41451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:17.210428 15125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:17.210714 15125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:17.210836 15125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:17.218248 15125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:17.218360 15125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:17.218502 15125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:27:17.228524 15125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:41451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:44713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:17.229830 15125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:17.231135 15125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:17.244093 15130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:17.244104 15133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:17.244130 15131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:17.245244 15125 server_base.cc:1034] running on GCE node
I20260618 22:27:17.245868 15125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:17.246802 15125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:17.248009 15125 hybrid_clock.cc:648] HybridClock initialized: now 1781821637247947 us; error 57 us; skew 500 ppm
I20260618 22:27:17.248349 15125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:17 dist-test-slave-0956 krb5kdc[14752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821637, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:17.251467 15125 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:27:17.252946 15125 webserver.cc:460] Webserver started at http://127.10.242.195:34487/ using document root <none> and password file <none>
I20260618 22:27:17.253822 15125 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:17.253945 15125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:17.254263 15125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:17.257309 15125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "66be2e1de2fd4a7f80e16d76431539d2"
format_stamp: "Formatted at 2026-06-18 22:27:17 on dist-test-slave-0956"
server_key: "c1f208694412aef966745849570b5f3f"
server_key_iv: "f8b91c43a6d7b038241e8513e0ebef36"
server_key_version: "encryptionkey@0"
I20260618 22:27:17.258100 15125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "66be2e1de2fd4a7f80e16d76431539d2"
format_stamp: "Formatted at 2026-06-18 22:27:17 on dist-test-slave-0956"
server_key: "c1f208694412aef966745849570b5f3f"
server_key_iv: "f8b91c43a6d7b038241e8513e0ebef36"
server_key_version: "encryptionkey@0"
I20260618 22:27:17.263250 15125 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:27:17.266722 15140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:17.268069 15125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:27:17.268215 15125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "66be2e1de2fd4a7f80e16d76431539d2"
format_stamp: "Formatted at 2026-06-18 22:27:17 on dist-test-slave-0956"
server_key: "c1f208694412aef966745849570b5f3f"
server_key_iv: "f8b91c43a6d7b038241e8513e0ebef36"
server_key_version: "encryptionkey@0"
I20260618 22:27:17.268399 15125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:17.279548 15125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:17.283515 15125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:17.283773 15125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:17.284637 15125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:17.285945 15125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:17.286018 15125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:17.286129 15125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:17.286180 15125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:17.315825 15125 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:43485
I20260618 22:27:17.315860 15252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:43485 every 8 connection(s)
I20260618 22:27:17.317400 15125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:27:17.320043 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 15125
I20260618 22:27:17.320220 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:27:17.320734 11211 external_mini_cluster.cc:1428] Setting key ebd822436e3884d34c5e72637d217515
Jun 18 22:27:17 dist-test-slave-0956 krb5kdc[14752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821637, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:17.336442 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:17.319736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:47453 (local address 127.10.242.254:44713)
0618 22:27:17.319891 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:17.319896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:17.321346 (+ 1450us) server_negotiation.cc:407] Connection header received
0618 22:27:17.323133 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:17.323138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:17.323216 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:17.323299 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:17.324721 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:17.325508 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:17.327032 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:17.327425 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:17.327529 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:17.330852 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:17.330856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:17.330860 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:17.330892 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:17.333196 (+ 2304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:17.333857 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:17.333863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:17.333866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:17.333978 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:17.334621 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:17.334627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:17.334630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:17.334947 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:17.335086 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:17.335810 (+ 724us) server_negotiation.cc:299] Negotiation successful
0618 22:27:17.336026 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260618 22:27:17.337410 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:17.320425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:44713 (local address 127.10.242.195:47453)
0618 22:27:17.321199 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:17.321241 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:17.322853 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:17.323523 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:17.323532 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:17.324006 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:17.324576 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:17.324593 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:17.325646 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:17.325650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:17.326907 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:17.326919 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:17.327630 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:17.327636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:17.327860 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:17.328435 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:17.328460 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:17.330630 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:17.333330 (+ 2700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:17.333339 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:17.333356 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:17.333750 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:17.334090 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:17.334166 (+ 76us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:17.334170 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:17.334383 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:17.335078 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:17.335087 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:17.335436 (+ 349us) client_negotiation.cc:769] Sending connection context
0618 22:27:17.335695 (+ 259us) client_negotiation.cc:240] Negotiation successful
0618 22:27:17.336250 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":310,"threads_started":1}
I20260618 22:27:17.338877 15253 heartbeater.cc:347] Connected to a master server at 127.10.242.254:44713
I20260618 22:27:17.339205 15253 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:17.339912 15253 heartbeater.cc:511] Master 127.10.242.254:44713 requested a full tablet report, sending...
I20260618 22:27:17.341370 14797 ts_manager.cc:194] Registered new tserver with Master: 66be2e1de2fd4a7f80e16d76431539d2 (127.10.242.195:43485)
I20260618 22:27:17.342119 14797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:47453
I20260618 22:27:17.347422 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 18 22:27:17 dist-test-slave-0956 krb5kdc[14752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 18 22:27:17 dist-test-slave-0956 krb5kdc[14752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:27:17.387957 14986 heartbeater.cc:503] Master 127.10.242.254:44713 was elected leader, sending a full tablet report...
I20260618 22:27:17.391595 15260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:17.377370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41742 (local address 127.10.242.193:42421)
0618 22:27:17.377911 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:17.377919 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:17.377998 (+ 79us) server_negotiation.cc:407] Connection header received
0618 22:27:17.378145 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:17.378152 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:17.378391 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:17.378519 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:17.379836 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:17.380799 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:17.382018 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:17.382581 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:17.382675 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:17.386310 (+ 3635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:17.386323 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:17.386338 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:17.386376 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:17.389150 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:17.389835 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:17.389843 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:17.389845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:17.389980 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:17.390289 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:17.390292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:17.390294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:17.390620 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:17.390841 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:17.391041 (+ 200us) server_negotiation.cc:299] Negotiation successful
0618 22:27:17.391243 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":251,"threads_started":1}
Jun 18 22:27:17 dist-test-slave-0956 krb5kdc[14752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:17.408754 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:17.395581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50676 (local address 127.10.242.254:44713)
0618 22:27:17.395791 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:17.395797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:17.395938 (+ 141us) server_negotiation.cc:407] Connection header received
0618 22:27:17.396136 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:17.396140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:17.396202 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:17.396283 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:17.397518 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:17.398233 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:17.399442 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:17.399855 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:17.399963 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:17.403600 (+ 3637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:17.403605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:17.403609 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:17.403643 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:17.406222 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:17.406922 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:17.406927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:17.406929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:17.407055 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:17.407398 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:17.407402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:17.407404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:17.407731 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:17.407945 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:17.408246 (+ 301us) server_negotiation.cc:299] Negotiation successful
0618 22:27:17.408444 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260618 22:27:17.409390 14797 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50676
I20260618 22:27:17.410467 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 14858
I20260618 22:27:17.422616 15260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:17.411413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41746 (local address 127.10.242.193:42421)
0618 22:27:17.411651 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:17.411657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:17.411736 (+ 79us) server_negotiation.cc:407] Connection header received
0618 22:27:17.411899 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:17.411905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:17.411968 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:17.412050 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:17.413270 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:17.414049 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:17.415326 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:17.415739 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:17.415847 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:17.418105 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:17.418111 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:17.418114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:17.418146 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:17.420258 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:17.420862 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:17.420866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:17.420868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:17.420985 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:17.421357 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:17.421362 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:17.421366 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:17.421613 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:17.421824 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:17.422097 (+ 273us) server_negotiation.cc:299] Negotiation successful
0618 22:27:17.422313 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260618 22:27:17.423288 14981 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:17.503141 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 14858
I20260618 22:27:17.515806 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 14992
Jun 18 22:27:17 dist-test-slave-0956 krb5kdc[14752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.194@KRBTEST.COM
I20260618 22:27:17.531096 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:17.516676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57884 (local address 127.10.242.194:43899)
0618 22:27:17.517206 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:17.517213 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:17.517237 (+ 24us) server_negotiation.cc:407] Connection header received
0618 22:27:17.517298 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:17.517305 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:17.517505 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:17.517632 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:17.518727 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:17.519708 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:17.520922 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:17.521588 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:17.521682 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:17.525461 (+ 3779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:17.525473 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:17.525487 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:17.525522 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:17.528391 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:17.529000 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:17.529007 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:17.529009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:17.529191 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:17.529511 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:17.529514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:17.529516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:17.529916 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:17.530183 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:17.530555 (+ 372us) server_negotiation.cc:299] Negotiation successful
0618 22:27:17.530803 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":247,"threads_started":1}
I20260618 22:27:17.532339 15115 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:17.613102 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 14992
I20260618 22:27:17.625226 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 15125
Jun 18 22:27:17 dist-test-slave-0956 krb5kdc[14752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.195@KRBTEST.COM
I20260618 22:27:17.641610 15267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:17.626145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33184 (local address 127.10.242.195:43485)
0618 22:27:17.626734 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:17.626741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:17.626771 (+ 30us) server_negotiation.cc:407] Connection header received
0618 22:27:17.626831 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:17.626837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:17.627043 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:17.627214 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:17.628481 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:17.629572 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:17.630895 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:17.631458 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:17.631572 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:17.636024 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:17.636039 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:17.636054 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:17.636101 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:17.638960 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:17.639680 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:17.639687 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:17.639689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:17.639811 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:17.640178 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:17.640181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:17.640183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:17.640529 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:17.640741 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:17.641029 (+ 288us) server_negotiation.cc:299] Negotiation successful
0618 22:27:17.641268 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":290,"threads_started":1}
I20260618 22:27:17.642813 15248 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:17.723805 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 15125
I20260618 22:27:17.735865 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 14765
I20260618 22:27:17.737015 14827 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:17.837767 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 14765
2026-06-18T22:27:17Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3645 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:27:17 dist-test-slave-0956 krb5kdc[15272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:27:17 dist-test-slave-0956 krb5kdc[15272](info): set up 2 sockets
Jun 18 22:27:17 dist-test-slave-0956 krb5kdc[15272](info): commencing operation
krb5kdc: starting...
W20260618 22:27:18.845423 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.971s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:27:18 dist-test-slave-0956 krb5kdc[15272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:27:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:27:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:27:19.038003 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:38745
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:46061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:38745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:19.357467 15285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:19.357825 15285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:19.357964 15285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:19.365603 15285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:27:19.365731 15285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:19.365789 15285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:19.365867 15285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:27:19.365923 15285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:27:19.376207 15285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:46061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:38745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:38745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:19.377594 15285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:19.378950 15285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:19.390717 15293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:19.390738 15290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:19.390693 15291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:19.391371 15285 server_base.cc:1034] running on GCE node
I20260618 22:27:19.392230 15285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:19.393671 15285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:19.394882 15285 hybrid_clock.cc:648] HybridClock initialized: now 1781821639394827 us; error 43 us; skew 500 ppm
I20260618 22:27:19.395241 15285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:19 dist-test-slave-0956 krb5kdc[15272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821639, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:19.404072 15285 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:27:19.405431 15285 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:19.405544 15285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:19.405858 15285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:19.408893 15285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "00113a73a8bc4487aa66152b381926e6"
format_stamp: "Formatted at 2026-06-18 22:27:19 on dist-test-slave-0956"
server_key: "7c2216f1c7c9becc5ffaa7feb8fe0b1c"
server_key_iv: "9483f3fcbec4bbb21684417d79a33de9"
server_key_version: "encryptionkey@0"
I20260618 22:27:19.409783 15285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "00113a73a8bc4487aa66152b381926e6"
format_stamp: "Formatted at 2026-06-18 22:27:19 on dist-test-slave-0956"
server_key: "7c2216f1c7c9becc5ffaa7feb8fe0b1c"
server_key_iv: "9483f3fcbec4bbb21684417d79a33de9"
server_key_version: "encryptionkey@0"
I20260618 22:27:19.415055 15285 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:27:19.418792 15299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:19.420236 15285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:27:19.420403 15285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "00113a73a8bc4487aa66152b381926e6"
format_stamp: "Formatted at 2026-06-18 22:27:19 on dist-test-slave-0956"
server_key: "7c2216f1c7c9becc5ffaa7feb8fe0b1c"
server_key_iv: "9483f3fcbec4bbb21684417d79a33de9"
server_key_version: "encryptionkey@0"
I20260618 22:27:19.420619 15285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:19.432221 15285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:19.436503 15285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:19.436772 15285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:19.453735 15350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:38745 every 8 connection(s)
I20260618 22:27:19.453747 15285 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:38745
I20260618 22:27:19.454691 15285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:27:19.456067 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 15285
I20260618 22:27:19.456265 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:27:19.457048 11211 external_mini_cluster.cc:1428] Setting key 56083cdbede394e675d08dd492d42136
I20260618 22:27:19.459204 15351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:19.470191 15351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6: Bootstrap starting.
Jun 18 22:27:19 dist-test-slave-0956 krb5kdc[15272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:19.473620 15351 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:19.474692 15351 log.cc:824] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:19.477423 15351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6: No bootstrap required, opened a new log
I20260618 22:27:19.479075 15354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:19.459184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37092 (local address 127.10.242.254:38745)
0618 22:27:19.460010 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:19.460022 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:19.460064 (+ 42us) server_negotiation.cc:407] Connection header received
0618 22:27:19.461747 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:19.461769 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:19.462179 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:19.462550 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:19.463992 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:19.465337 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:19.466546 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:19.467261 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:19.467427 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:19.471531 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:19.471540 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:19.471555 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:19.471599 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:19.474762 (+ 3163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:19.475589 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:19.475597 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:19.475607 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:19.475791 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:19.476192 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:19.476197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:19.476199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:19.476756 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:19.477039 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:19.477562 (+ 523us) server_negotiation.cc:299] Negotiation successful
0618 22:27:19.478003 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":406,"threads_started":1}
I20260618 22:27:19.485998 15351 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00113a73a8bc4487aa66152b381926e6" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 38745 } }
I20260618 22:27:19.486389 15351 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:19.486485 15351 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00113a73a8bc4487aa66152b381926e6, State: Initialized, Role: FOLLOWER
I20260618 22:27:19.487020 15351 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00113a73a8bc4487aa66152b381926e6" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 38745 } }
I20260618 22:27:19.487238 15351 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:27:19.487355 15351 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:27:19.487506 15351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:19.489918 15351 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00113a73a8bc4487aa66152b381926e6" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 38745 } }
I20260618 22:27:19.490411 15351 leader_election.cc:302] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00113a73a8bc4487aa66152b381926e6; no voters:
I20260618 22:27:19.491003 15351 leader_election.cc:288] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:27:19.491096 15356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:19.491760 15356 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6 [term 1 LEADER]: Becoming Leader. State: Replica: 00113a73a8bc4487aa66152b381926e6, State: Running, Role: LEADER
I20260618 22:27:19.492259 15356 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00113a73a8bc4487aa66152b381926e6" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 38745 } }
I20260618 22:27:19.492990 15351 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:27:19.495294 15358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00113a73a8bc4487aa66152b381926e6. Latest consensus state: current_term: 1 leader_uuid: "00113a73a8bc4487aa66152b381926e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00113a73a8bc4487aa66152b381926e6" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 38745 } } }
I20260618 22:27:19.495254 15357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00113a73a8bc4487aa66152b381926e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00113a73a8bc4487aa66152b381926e6" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 38745 } } }
I20260618 22:27:19.495700 15357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:19.495677 15358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:19.498564 15363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:27:19.504218 15363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:27:19.512899 15363 catalog_manager.cc:1347] Generated new cluster ID: 4faf8437630343909ae6ad0c46b6a99f
I20260618 22:27:19.513041 15363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:27:19.532728 15363 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:27:19.534152 15363 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:27:19.542243 15363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6: Generated new TSK 0
I20260618 22:27:19.543085 15363 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
I20260618 22:27:19.592360 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:38745
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:46061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:19.932092 15377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:19.932397 15377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:19.932518 15377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:19.939741 15377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:19.939873 15377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:19.940055 15377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:27:19.949918 15377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:46061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:38745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:19.951187 15377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:19.952505 15377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:19.965346 15383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:19.965276 15382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:19.965322 15385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:19.970683 15377 server_base.cc:1034] running on GCE node
I20260618 22:27:19.971326 15377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:19.972355 15377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:19.973551 15377 hybrid_clock.cc:648] HybridClock initialized: now 1781821639973493 us; error 54 us; skew 500 ppm
I20260618 22:27:19.973871 15377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:19 dist-test-slave-0956 krb5kdc[15272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821639, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:19.977118 15377 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:27:19.978338 15377 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:19.978442 15377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:19.978807 15377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:19.981855 15377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "fb758d2f58e84da3b598515ed0989b15"
format_stamp: "Formatted at 2026-06-18 22:27:19 on dist-test-slave-0956"
server_key: "196b865d6e2bca2f4196fa090e98f88d"
server_key_iv: "bd5abf03c3371e7bdbb0771ee3ce4b29"
server_key_version: "encryptionkey@0"
I20260618 22:27:19.982666 15377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "fb758d2f58e84da3b598515ed0989b15"
format_stamp: "Formatted at 2026-06-18 22:27:19 on dist-test-slave-0956"
server_key: "196b865d6e2bca2f4196fa090e98f88d"
server_key_iv: "bd5abf03c3371e7bdbb0771ee3ce4b29"
server_key_version: "encryptionkey@0"
I20260618 22:27:19.987718 15377 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:27:19.991393 15391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:19.992956 15377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:27:19.993222 15377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "fb758d2f58e84da3b598515ed0989b15"
format_stamp: "Formatted at 2026-06-18 22:27:19 on dist-test-slave-0956"
server_key: "196b865d6e2bca2f4196fa090e98f88d"
server_key_iv: "bd5abf03c3371e7bdbb0771ee3ce4b29"
server_key_version: "encryptionkey@0"
I20260618 22:27:19.993445 15377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:20.014631 15377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:20.019109 15377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:20.019387 15377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:20.020429 15377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:20.021898 15377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:20.021998 15377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:20.022117 15377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:20.022185 15377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:20.053596 15377 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:39401
I20260618 22:27:20.053666 15503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:39401 every 8 connection(s)
I20260618 22:27:20.054445 15377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:27:20.063676 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 15377
I20260618 22:27:20.063862 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:27:20.064455 11211 external_mini_cluster.cc:1428] Setting key 3341ac774401e0056bbcd02324b2d2a7
Jun 18 22:27:20 dist-test-slave-0956 krb5kdc[15272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821639, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:20.074676 15506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:20.057213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:49221 (local address 127.10.242.254:38745)
0618 22:27:20.057599 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:20.057604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:20.058623 (+ 1019us) server_negotiation.cc:407] Connection header received
0618 22:27:20.060466 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:20.060472 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:20.060555 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:20.060651 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:20.062310 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:20.063011 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:20.064724 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:20.065283 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:20.065408 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:20.069055 (+ 3647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:20.069060 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:20.069064 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:20.069136 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:20.071566 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:20.072316 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:20.072320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:20.072323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:20.072440 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:20.072858 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:20.072863 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:20.072866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:20.073197 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:20.073324 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:20.074056 (+ 732us) server_negotiation.cc:299] Negotiation successful
0618 22:27:20.074296 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":187,"threads_started":1}
I20260618 22:27:20.075703 15507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:20.057728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:38745 (local address 127.10.242.193:49221)
0618 22:27:20.058469 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:20.058508 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:20.060172 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:20.060900 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:20.060909 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:20.061534 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:20.062124 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:20.062142 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:20.063170 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:20.063174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:20.064524 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:20.064538 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:20.065521 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:20.065529 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:20.065779 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:20.066421 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:20.066443 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:20.068792 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:20.071736 (+ 2944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:20.071747 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:20.071763 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:20.072198 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:20.072544 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:20.072550 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:20.072554 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:20.072752 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:20.073323 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:20.073331 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:20.073700 (+ 369us) client_negotiation.cc:769] Sending connection context
0618 22:27:20.073953 (+ 253us) client_negotiation.cc:240] Negotiation successful
0618 22:27:20.074522 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":262,"threads_started":1}
I20260618 22:27:20.077399 15504 heartbeater.cc:347] Connected to a master server at 127.10.242.254:38745
I20260618 22:27:20.077754 15504 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:20.078629 15504 heartbeater.cc:511] Master 127.10.242.254:38745 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.242.194@KRBTEST.COM; defaulting to no policy
I20260618 22:27:20.081351 15316 ts_manager.cc:194] Registered new tserver with Master: fb758d2f58e84da3b598515ed0989b15 (127.10.242.193:39401)
I20260618 22:27:20.082934 15316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:49221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
I20260618 22:27:20.105886 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:38745
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:46061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:20.421887 15510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:20.422191 15510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:20.422288 15510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:20.429672 15510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:20.429798 15510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:20.429914 15510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:27:20.439646 15510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:46061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:38745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:20.440925 15510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:20.442268 15510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:20.454648 15516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:20.454614 15515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:20.455039 15518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:20.456101 15510 server_base.cc:1034] running on GCE node
I20260618 22:27:20.456682 15510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:20.457693 15510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:20.458904 15510 hybrid_clock.cc:648] HybridClock initialized: now 1781821640458860 us; error 39 us; skew 500 ppm
I20260618 22:27:20.459211 15510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:20 dist-test-slave-0956 krb5kdc[15272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821640, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:20.462312 15510 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:27:20.463445 15510 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:20.463531 15510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:20.463824 15510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:20.466717 15510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "cc09c571c1ad4057bb0b8e5a64dd1c2d"
format_stamp: "Formatted at 2026-06-18 22:27:20 on dist-test-slave-0956"
server_key: "d1c3590c027f921bab4381ba40530d77"
server_key_iv: "05963ca22b49007a492847ae0a6d243f"
server_key_version: "encryptionkey@0"
I20260618 22:27:20.467473 15510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "cc09c571c1ad4057bb0b8e5a64dd1c2d"
format_stamp: "Formatted at 2026-06-18 22:27:20 on dist-test-slave-0956"
server_key: "d1c3590c027f921bab4381ba40530d77"
server_key_iv: "05963ca22b49007a492847ae0a6d243f"
server_key_version: "encryptionkey@0"
I20260618 22:27:20.472448 15510 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260618 22:27:20.476192 15524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:20.477600 15510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:27:20.477762 15510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "cc09c571c1ad4057bb0b8e5a64dd1c2d"
format_stamp: "Formatted at 2026-06-18 22:27:20 on dist-test-slave-0956"
server_key: "d1c3590c027f921bab4381ba40530d77"
server_key_iv: "05963ca22b49007a492847ae0a6d243f"
server_key_version: "encryptionkey@0"
I20260618 22:27:20.477962 15510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:20.498634 15510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:20.502930 15510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:20.503193 15510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:20.504155 15510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:20.505545 15510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:20.505635 15510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:20.505753 15510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:20.505815 15510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:20.535581 15510 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:37691
I20260618 22:27:20.535676 15636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:37691 every 8 connection(s)
I20260618 22:27:20.536401 15510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:27:20.544067 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 15510
I20260618 22:27:20.544229 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:27:20.544643 11211 external_mini_cluster.cc:1428] Setting key fbe973262855b8318169ab906a79275d
Jun 18 22:27:20 dist-test-slave-0956 krb5kdc[15272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821640, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:20.556332 15506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:20.538934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:40735 (local address 127.10.242.254:38745)
0618 22:27:20.539100 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:20.539105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:20.540230 (+ 1125us) server_negotiation.cc:407] Connection header received
0618 22:27:20.542085 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:20.542091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:20.542172 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:20.542257 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:20.543761 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:20.544507 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:20.546269 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:20.546773 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:20.546904 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:20.550490 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:20.550495 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:20.550499 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:20.550532 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:20.553213 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:20.554063 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:20.554067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:20.554069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:20.554207 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:20.554612 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:20.554615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:20.554617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:20.554869 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:20.554976 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:20.555804 (+ 828us) server_negotiation.cc:299] Negotiation successful
0618 22:27:20.556007 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:27:20.557653 15639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:20.539351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:38745 (local address 127.10.242.194:40735)
0618 22:27:20.540053 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:20.540098 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:20.541789 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:20.542504 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:20.542513 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:20.543032 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:20.543594 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:20.543611 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:20.544644 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:20.544648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:20.546114 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:20.546126 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:20.546967 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:20.546971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:20.547156 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:20.547767 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:20.547789 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:20.550247 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:20.553347 (+ 3100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:20.553356 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:20.553373 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:20.553953 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:20.554297 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:20.554301 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:20.554304 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:20.554515 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:20.554983 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:20.554991 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:20.555396 (+ 405us) client_negotiation.cc:769] Sending connection context
0618 22:27:20.555708 (+ 312us) client_negotiation.cc:240] Negotiation successful
0618 22:27:20.556284 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":295,"threads_started":1}
I20260618 22:27:20.559185 15637 heartbeater.cc:347] Connected to a master server at 127.10.242.254:38745
I20260618 22:27:20.559487 15637 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:20.560196 15637 heartbeater.cc:511] Master 127.10.242.254:38745 requested a full tablet report, sending...
I20260618 22:27:20.561867 15316 ts_manager.cc:194] Registered new tserver with Master: cc09c571c1ad4057bb0b8e5a64dd1c2d (127.10.242.194:37691)
I20260618 22:27:20.562987 15316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:40735
WARNING: no policy specified for kudu/127.10.242.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
I20260618 22:27:20.587598 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:38745
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:46061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:20.904093 15642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:20.904379 15642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:20.904475 15642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:20.911644 15642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:20.911774 15642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:20.911890 15642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:27:20.921609 15642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:46061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:38745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:20.922878 15642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:20.924219 15642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:20.936947 15647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:20.936965 15648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:20.937232 15650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:20.937841 15642 server_base.cc:1034] running on GCE node
I20260618 22:27:20.938477 15642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:20.939481 15642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:20.940732 15642 hybrid_clock.cc:648] HybridClock initialized: now 1781821640940656 us; error 68 us; skew 500 ppm
I20260618 22:27:20.941114 15642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:20 dist-test-slave-0956 krb5kdc[15272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821640, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:20.944294 15642 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:27:20.945530 15642 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:20.945632 15642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:20.945922 15642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:20.949131 15642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "127d235663484d089e619d73b8d7e2e3"
format_stamp: "Formatted at 2026-06-18 22:27:20 on dist-test-slave-0956"
server_key: "280e61cd85b67818f754aaef2389fee5"
server_key_iv: "d736c4a1665ae9a04195566ad7c2195b"
server_key_version: "encryptionkey@0"
I20260618 22:27:20.949977 15642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "127d235663484d089e619d73b8d7e2e3"
format_stamp: "Formatted at 2026-06-18 22:27:20 on dist-test-slave-0956"
server_key: "280e61cd85b67818f754aaef2389fee5"
server_key_iv: "d736c4a1665ae9a04195566ad7c2195b"
server_key_version: "encryptionkey@0"
I20260618 22:27:20.955396 15642 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260618 22:27:20.959400 15656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:20.960968 15642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:27:20.961186 15642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "127d235663484d089e619d73b8d7e2e3"
format_stamp: "Formatted at 2026-06-18 22:27:20 on dist-test-slave-0956"
server_key: "280e61cd85b67818f754aaef2389fee5"
server_key_iv: "d736c4a1665ae9a04195566ad7c2195b"
server_key_version: "encryptionkey@0"
I20260618 22:27:20.961486 15642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:20.978238 15642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:20.982590 15642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:20.982865 15642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:20.983815 15642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:20.985201 15642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:20.985296 15642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:20.985399 15642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:20.985471 15642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:21.014243 15642 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:38709
I20260618 22:27:21.014289 15768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:38709 every 8 connection(s)
I20260618 22:27:21.015089 15642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:27:21.015735 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 15642
I20260618 22:27:21.015898 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:27:21.016381 11211 external_mini_cluster.cc:1428] Setting key 02244be7af9c5232dd7e80c509a3d4cf
Jun 18 22:27:21 dist-test-slave-0956 krb5kdc[15272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821640, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:21.035133 15506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.017545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:52705 (local address 127.10.242.254:38745)
0618 22:27:21.017724 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:21.017730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:21.019050 (+ 1320us) server_negotiation.cc:407] Connection header received
0618 22:27:21.020795 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:21.020799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:21.020865 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:21.020962 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:21.022555 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.023357 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.025205 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.025657 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.025757 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:21.029024 (+ 3267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:21.029029 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:21.029032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:21.029065 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:21.032107 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:21.032894 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:21.032898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:21.032901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:21.033031 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:21.033507 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:21.033511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:21.033513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:21.033790 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:21.033896 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:21.034704 (+ 808us) server_negotiation.cc:299] Negotiation successful
0618 22:27:21.034874 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260618 22:27:21.036069 15771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.018143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:38745 (local address 127.10.242.195:52705)
0618 22:27:21.018894 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:21.018943 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:21.020546 (+ 1603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:21.021251 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:21.021261 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:21.021771 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:21.022371 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:21.022395 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.023539 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.023543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:21.024973 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:21.024985 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.025844 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.025848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:21.026032 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:21.026706 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:21.026727 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:21.028804 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:21.032279 (+ 3475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:21.032287 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:21.032303 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:21.032761 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:21.033231 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:21.033235 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:21.033237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:21.033415 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:21.033946 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:21.033953 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:21.034334 (+ 381us) client_negotiation.cc:769] Sending connection context
0618 22:27:21.034584 (+ 250us) client_negotiation.cc:240] Negotiation successful
0618 22:27:21.034987 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":290,"threads_started":1}
I20260618 22:27:21.037410 15769 heartbeater.cc:347] Connected to a master server at 127.10.242.254:38745
I20260618 22:27:21.037668 15769 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:21.038326 15769 heartbeater.cc:511] Master 127.10.242.254:38745 requested a full tablet report, sending...
I20260618 22:27:21.039664 15316 ts_manager.cc:194] Registered new tserver with Master: 127d235663484d089e619d73b8d7e2e3 (127.10.242.195:38709)
I20260618 22:27:21.040416 15316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:52705
I20260618 22:27:21.044325 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:27:21.061259 15506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.048666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37102 (local address 127.10.242.254:38745)
0618 22:27:21.048819 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:21.048824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:21.049328 (+ 504us) server_negotiation.cc:407] Connection header received
0618 22:27:21.049501 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:21.049505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:21.049565 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:21.049645 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:21.051025 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.051756 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.053025 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.053500 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.053610 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:21.055957 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:21.055963 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:21.055966 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:21.056002 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:21.058622 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:21.059360 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:21.059365 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:21.059369 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:21.059514 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:21.059860 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:21.059865 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:21.059868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:21.060180 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:21.060405 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:21.060719 (+ 314us) server_negotiation.cc:299] Negotiation successful
0618 22:27:21.060931 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:27:21.065337 15316 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:27:21.072011 15316 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:27:21.086787 15504 heartbeater.cc:503] Master 127.10.242.254:38745 was elected leader, sending a full tablet report...
I20260618 22:27:21.109071 15785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.086368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50564 (local address 127.10.242.193:39401)
0618 22:27:21.100364 (+ 13996us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:21.100374 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:21.100407 (+ 33us) server_negotiation.cc:407] Connection header received
0618 22:27:21.100522 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:21.100531 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:21.100857 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:21.101134 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:21.102049 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.103588 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.106340 (+ 2752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.107825 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.108029 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:21.108212 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:21.108532 (+ 320us) server_negotiation.cc:299] Negotiation successful
0618 22:27:21.108703 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13826,"thread_start_us":13690,"threads_started":1}
I20260618 22:27:21.109153 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.085986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39401 (local address 127.0.0.1:50564)
0618 22:27:21.099520 (+ 13534us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:21.099541 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:21.099689 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:21.101470 (+ 1781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:21.101479 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:21.101529 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:21.101883 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:21.101897 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.103784 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.103789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:21.106156 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:21.106167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.108041 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.108047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:21.108254 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:21.108311 (+ 57us) client_negotiation.cc:769] Sending connection context
0618 22:27:21.108447 (+ 136us) client_negotiation.cc:240] Negotiation successful
0618 22:27:21.108657 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13383,"mutex_wait_us":26,"thread_start_us":13771,"threads_started":1}
I20260618 22:27:21.109640 15781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.084603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:37691 (local address 127.0.0.1:58410)
0618 22:27:21.085493 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:21.085546 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:21.085737 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:21.103148 (+ 17411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:21.103153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:21.103188 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:21.103519 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:21.103530 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.105308 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.105314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:21.107164 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:21.107175 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.108654 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.108660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:21.108924 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:21.108954 (+ 30us) client_negotiation.cc:769] Sending connection context
0618 22:27:21.109035 (+ 81us) client_negotiation.cc:240] Negotiation successful
0618 22:27:21.109212 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":29,"thread_start_us":419,"threads_started":1}
I20260618 22:27:21.109663 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.085061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:38709 (local address 127.0.0.1:49854)
0618 22:27:21.099201 (+ 14140us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:21.099220 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:21.099354 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:21.102752 (+ 3398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:21.102756 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:21.102782 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:21.103149 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:21.103159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.104625 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.104630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:21.106640 (+ 2010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:21.106655 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.108633 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.108640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:21.108868 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:21.108926 (+ 58us) client_negotiation.cc:769] Sending connection context
0618 22:27:21.109014 (+ 88us) client_negotiation.cc:240] Negotiation successful
0618 22:27:21.109187 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13979,"mutex_wait_us":36,"thread_start_us":13321,"threads_started":1}
I20260618 22:27:21.110466 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.084753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58410 (local address 127.10.242.194:37691)
0618 22:27:21.102558 (+ 17805us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:21.102565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:21.102592 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:27:21.102676 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:21.102682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:21.102919 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:21.103149 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:21.103667 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.105154 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.107305 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.108387 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.109665 (+ 1278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:21.109987 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:21.110054 (+ 67us) server_negotiation.cc:299] Negotiation successful
0618 22:27:21.110201 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17637,"thread_start_us":18095,"threads_started":1}
I20260618 22:27:21.110623 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.086350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49854 (local address 127.10.242.195:38709)
0618 22:27:21.101874 (+ 15524us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:21.101884 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:21.101919 (+ 35us) server_negotiation.cc:407] Connection header received
0618 22:27:21.102025 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:21.102035 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:21.102289 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:21.102481 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:21.103393 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.104493 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.106785 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.108146 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.108325 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:21.108457 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:21.109748 (+ 1291us) server_negotiation.cc:299] Negotiation successful
0618 22:27:21.110293 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15341,"thread_start_us":15157,"threads_started":1}
I20260618 22:27:21.113689 15572 tablet_service.cc:1467] Processing CreateTablet for tablet eda727675b0748eb8d0c131a5c09a3ff (DEFAULT_TABLE table=test-table [id=22077ec6aaa548988400b780e748bb41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:21.113787 15439 tablet_service.cc:1467] Processing CreateTablet for tablet eda727675b0748eb8d0c131a5c09a3ff (DEFAULT_TABLE table=test-table [id=22077ec6aaa548988400b780e748bb41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:21.113777 15704 tablet_service.cc:1467] Processing CreateTablet for tablet eda727675b0748eb8d0c131a5c09a3ff (DEFAULT_TABLE table=test-table [id=22077ec6aaa548988400b780e748bb41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:21.114964 15572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eda727675b0748eb8d0c131a5c09a3ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:21.115046 15704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eda727675b0748eb8d0c131a5c09a3ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:21.115105 15439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eda727675b0748eb8d0c131a5c09a3ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:21.126557 15787 tablet_bootstrap.cc:492] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d: Bootstrap starting.
I20260618 22:27:21.130148 15787 tablet_bootstrap.cc:654] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:21.131325 15787 log.cc:824] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:21.134418 15787 tablet_bootstrap.cc:492] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d: No bootstrap required, opened a new log
I20260618 22:27:21.134708 15787 ts_tablet_manager.cc:1397] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260618 22:27:21.137900 15788 tablet_bootstrap.cc:492] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15: Bootstrap starting.
I20260618 22:27:21.138239 15789 tablet_bootstrap.cc:492] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3: Bootstrap starting.
I20260618 22:27:21.141430 15789 tablet_bootstrap.cc:654] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:21.142204 15787 raft_consensus.cc:348] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc09c571c1ad4057bb0b8e5a64dd1c2d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37691 } } peers { permanent_uuid: "fb758d2f58e84da3b598515ed0989b15" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39401 } } peers { permanent_uuid: "127d235663484d089e619d73b8d7e2e3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 38709 } }
I20260618 22:27:21.142565 15787 raft_consensus.cc:374] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:21.142686 15787 raft_consensus.cc:729] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc09c571c1ad4057bb0b8e5a64dd1c2d, State: Initialized, Role: FOLLOWER
I20260618 22:27:21.142745 15789 log.cc:824] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:21.143241 15787 consensus_queue.cc:260] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc09c571c1ad4057bb0b8e5a64dd1c2d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37691 } } peers { permanent_uuid: "fb758d2f58e84da3b598515ed0989b15" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39401 } } peers { permanent_uuid: "127d235663484d089e619d73b8d7e2e3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 38709 } }
I20260618 22:27:21.144917 15787 ts_tablet_manager.cc:1428] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260618 22:27:21.145020 15788 tablet_bootstrap.cc:654] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:21.145371 15637 heartbeater.cc:503] Master 127.10.242.254:38745 was elected leader, sending a full tablet report...
I20260618 22:27:21.146171 15789 tablet_bootstrap.cc:492] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3: No bootstrap required, opened a new log
I20260618 22:27:21.146447 15789 ts_tablet_manager.cc:1397] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.000s
I20260618 22:27:21.146409 15788 log.cc:824] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:21.149659 15788 tablet_bootstrap.cc:492] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15: No bootstrap required, opened a new log
I20260618 22:27:21.149931 15788 ts_tablet_manager.cc:1397] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.005s
I20260618 22:27:21.156471 15789 raft_consensus.cc:348] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc09c571c1ad4057bb0b8e5a64dd1c2d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37691 } } peers { permanent_uuid: "fb758d2f58e84da3b598515ed0989b15" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39401 } } peers { permanent_uuid: "127d235663484d089e619d73b8d7e2e3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 38709 } }
I20260618 22:27:21.156944 15789 raft_consensus.cc:374] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:21.157042 15789 raft_consensus.cc:729] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 127d235663484d089e619d73b8d7e2e3, State: Initialized, Role: FOLLOWER
I20260618 22:27:21.157727 15789 consensus_queue.cc:260] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc09c571c1ad4057bb0b8e5a64dd1c2d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37691 } } peers { permanent_uuid: "fb758d2f58e84da3b598515ed0989b15" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39401 } } peers { permanent_uuid: "127d235663484d089e619d73b8d7e2e3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 38709 } }
I20260618 22:27:21.158826 15769 heartbeater.cc:503] Master 127.10.242.254:38745 was elected leader, sending a full tablet report...
I20260618 22:27:21.159507 15789 ts_tablet_manager.cc:1428] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260618 22:27:21.159750 15788 raft_consensus.cc:348] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc09c571c1ad4057bb0b8e5a64dd1c2d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37691 } } peers { permanent_uuid: "fb758d2f58e84da3b598515ed0989b15" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39401 } } peers { permanent_uuid: "127d235663484d089e619d73b8d7e2e3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 38709 } }
I20260618 22:27:21.160148 15788 raft_consensus.cc:374] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:21.160252 15788 raft_consensus.cc:729] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb758d2f58e84da3b598515ed0989b15, State: Initialized, Role: FOLLOWER
I20260618 22:27:21.160795 15788 consensus_queue.cc:260] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc09c571c1ad4057bb0b8e5a64dd1c2d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37691 } } peers { permanent_uuid: "fb758d2f58e84da3b598515ed0989b15" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39401 } } peers { permanent_uuid: "127d235663484d089e619d73b8d7e2e3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 38709 } }
I20260618 22:27:21.162431 15788 ts_tablet_manager.cc:1428] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15: Time spent starting tablet: real 0.012s user 0.010s sys 0.003s
W20260618 22:27:21.266577 15770 tablet.cc:2369] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:27:21.288261 15638 tablet.cc:2369] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:27:21.307098 15505 tablet.cc:2369] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:27:21.359825 15794 raft_consensus.cc:482] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:27:21.360201 15794 raft_consensus.cc:504] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc09c571c1ad4057bb0b8e5a64dd1c2d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37691 } } peers { permanent_uuid: "fb758d2f58e84da3b598515ed0989b15" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39401 } } peers { permanent_uuid: "127d235663484d089e619d73b8d7e2e3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 38709 } }
I20260618 22:27:21.362054 15794 leader_election.cc:288] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc09c571c1ad4057bb0b8e5a64dd1c2d (127.10.242.194:37691), fb758d2f58e84da3b598515ed0989b15 (127.10.242.193:39401)
I20260618 22:27:21.368391 15771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.362360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:37691 (local address 127.10.242.195:54501)
0618 22:27:21.362583 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:21.362603 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:21.362745 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:21.363268 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:21.363274 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:21.363300 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:21.363505 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:21.363515 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.364765 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.364770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:21.366680 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:21.366695 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.367838 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.367842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:21.367979 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:21.368004 (+ 25us) client_negotiation.cc:769] Sending connection context
0618 22:27:21.368087 (+ 83us) client_negotiation.cc:240] Negotiation successful
0618 22:27:21.368187 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":18}
I20260618 22:27:21.368491 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.362591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:54501 (local address 127.10.242.194:37691)
0618 22:27:21.362810 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:21.362817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:21.362841 (+ 24us) server_negotiation.cc:407] Connection header received
0618 22:27:21.362920 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:21.362925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:21.363004 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:21.363134 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:21.363658 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.364586 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.366856 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.367680 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.367847 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:21.367919 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:21.368123 (+ 204us) server_negotiation.cc:299] Negotiation successful
0618 22:27:21.368248 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260618 22:27:21.369555 15785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.362980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:42321 (local address 127.10.242.193:39401)
0618 22:27:21.363162 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:21.363167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:21.363480 (+ 313us) server_negotiation.cc:407] Connection header received
0618 22:27:21.363620 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:21.363626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:21.363704 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:21.363810 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:21.364381 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.365271 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.367651 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.368728 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.368886 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:21.368950 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:21.369303 (+ 353us) server_negotiation.cc:299] Negotiation successful
0618 22:27:21.369396 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260618 22:27:21.369722 15796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.362812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39401 (local address 127.10.242.195:42321)
0618 22:27:21.363374 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:21.363390 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:21.363502 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:21.363894 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:21.363898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:21.363920 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:21.364222 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:21.364230 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.365418 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.365446 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:21.367477 (+ 2031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:21.367489 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.368911 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.368915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:21.369056 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:21.369116 (+ 60us) client_negotiation.cc:769] Sending connection context
0618 22:27:21.369271 (+ 155us) client_negotiation.cc:240] Negotiation successful
0618 22:27:21.369387 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":23,"thread_start_us":292,"threads_started":1}
I20260618 22:27:21.369526 15592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eda727675b0748eb8d0c131a5c09a3ff" candidate_uuid: "127d235663484d089e619d73b8d7e2e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc09c571c1ad4057bb0b8e5a64dd1c2d" is_pre_election: true
I20260618 22:27:21.370116 15592 raft_consensus.cc:2454] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 127d235663484d089e619d73b8d7e2e3 in term 0.
I20260618 22:27:21.370811 15657 leader_election.cc:302] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 127d235663484d089e619d73b8d7e2e3, cc09c571c1ad4057bb0b8e5a64dd1c2d; no voters:
I20260618 22:27:21.370817 15459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eda727675b0748eb8d0c131a5c09a3ff" candidate_uuid: "127d235663484d089e619d73b8d7e2e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb758d2f58e84da3b598515ed0989b15" is_pre_election: true
I20260618 22:27:21.371291 15794 raft_consensus.cc:2781] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:27:21.371349 15459 raft_consensus.cc:2454] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 127d235663484d089e619d73b8d7e2e3 in term 0.
I20260618 22:27:21.371413 15794 raft_consensus.cc:482] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:27:21.371500 15794 raft_consensus.cc:3037] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:21.373752 15794 raft_consensus.cc:504] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc09c571c1ad4057bb0b8e5a64dd1c2d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37691 } } peers { permanent_uuid: "fb758d2f58e84da3b598515ed0989b15" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39401 } } peers { permanent_uuid: "127d235663484d089e619d73b8d7e2e3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 38709 } }
I20260618 22:27:21.374330 15794 leader_election.cc:288] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [CANDIDATE]: Term 1 election: Requested vote from peers cc09c571c1ad4057bb0b8e5a64dd1c2d (127.10.242.194:37691), fb758d2f58e84da3b598515ed0989b15 (127.10.242.193:39401)
I20260618 22:27:21.374730 15592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eda727675b0748eb8d0c131a5c09a3ff" candidate_uuid: "127d235663484d089e619d73b8d7e2e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc09c571c1ad4057bb0b8e5a64dd1c2d"
I20260618 22:27:21.374815 15459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eda727675b0748eb8d0c131a5c09a3ff" candidate_uuid: "127d235663484d089e619d73b8d7e2e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb758d2f58e84da3b598515ed0989b15"
I20260618 22:27:21.374912 15592 raft_consensus.cc:3037] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:21.375018 15459 raft_consensus.cc:3037] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:21.377022 15592 raft_consensus.cc:2454] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 127d235663484d089e619d73b8d7e2e3 in term 1.
I20260618 22:27:21.377053 15459 raft_consensus.cc:2454] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 127d235663484d089e619d73b8d7e2e3 in term 1.
I20260618 22:27:21.377388 15657 leader_election.cc:302] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 127d235663484d089e619d73b8d7e2e3, cc09c571c1ad4057bb0b8e5a64dd1c2d; no voters:
I20260618 22:27:21.377635 15794 raft_consensus.cc:2781] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:21.378026 15794 raft_consensus.cc:686] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [term 1 LEADER]: Becoming Leader. State: Replica: 127d235663484d089e619d73b8d7e2e3, State: Running, Role: LEADER
I20260618 22:27:21.378469 15794 consensus_queue.cc:237] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc09c571c1ad4057bb0b8e5a64dd1c2d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37691 } } peers { permanent_uuid: "fb758d2f58e84da3b598515ed0989b15" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39401 } } peers { permanent_uuid: "127d235663484d089e619d73b8d7e2e3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 38709 } }
I20260618 22:27:21.382863 15315 catalog_manager.cc:5495] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 127d235663484d089e619d73b8d7e2e3 (127.10.242.195). New cstate: current_term: 1 leader_uuid: "127d235663484d089e619d73b8d7e2e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc09c571c1ad4057bb0b8e5a64dd1c2d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb758d2f58e84da3b598515ed0989b15" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "127d235663484d089e619d73b8d7e2e3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 38709 } health_report { overall_health: HEALTHY } } }
I20260618 22:27:21.404879 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.399908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49864 (local address 127.10.242.195:38709)
0618 22:27:21.400119 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:21.400125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:21.400144 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:27:21.400237 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:21.400241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:21.400312 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:21.400470 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:27:21.400822 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.401569 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.402839 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.403224 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.403341 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:21.403817 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:27:21.404310 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:27:21.404474 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:21.404547 (+ 73us) server_negotiation.cc:299] Negotiation successful
0618 22:27:21.404629 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:27:21.413214 15459 raft_consensus.cc:1261] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15 [term 1 FOLLOWER]: Refusing update from remote peer 127d235663484d089e619d73b8d7e2e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:21.413311 15592 raft_consensus.cc:1261] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d [term 1 FOLLOWER]: Refusing update from remote peer 127d235663484d089e619d73b8d7e2e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:21.414369 15797 consensus_queue.cc:1035] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc09c571c1ad4057bb0b8e5a64dd1c2d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:21.414752 15794 consensus_queue.cc:1035] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb758d2f58e84da3b598515ed0989b15" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:21.426054 15800 mvcc.cc:205] Tried to move back new op lower bound from 7298341443217723392 to 7298341443089715200. Current Snapshot: MvccSnapshot[applied={T|T < 7298341443217723392}]
I20260618 22:27:21.426292 15802 mvcc.cc:205] Tried to move back new op lower bound from 7298341443217723392 to 7298341443089715200. Current Snapshot: MvccSnapshot[applied={T|T < 7298341443217723392}]
I20260618 22:27:21.440975 15315 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:21.441332 15315 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:21.445284 15315 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00113a73a8bc4487aa66152b381926e6: Sending DeleteTablet for 3 replicas of tablet eda727675b0748eb8d0c131a5c09a3ff
I20260618 22:27:21.446527 15704 tablet_service.cc:1514] Processing DeleteTablet for tablet eda727675b0748eb8d0c131a5c09a3ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:21 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:49854
I20260618 22:27:21.446647 15439 tablet_service.cc:1514] Processing DeleteTablet for tablet eda727675b0748eb8d0c131a5c09a3ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:21 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:50564
I20260618 22:27:21.446940 15572 tablet_service.cc:1514] Processing DeleteTablet for tablet eda727675b0748eb8d0c131a5c09a3ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:21 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:58410
I20260618 22:27:21.448906 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 15377
I20260618 22:27:21.493638 15812 tablet_replica.cc:330] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d: stopping tablet replica
I20260618 22:27:21.494246 15812 raft_consensus.cc:2229] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:21.495415 15811 tablet_replica.cc:330] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3: stopping tablet replica
I20260618 22:27:21.495875 15811 raft_consensus.cc:2229] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:27:21.496564 15811 raft_consensus.cc:2258] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:21.497575 15812 raft_consensus.cc:2258] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:21.500279 15812 ts_tablet_manager.cc:1905] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:21.502254 15811 ts_tablet_manager.cc:1905] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 18 22:27:21 dist-test-slave-0956 krb5kdc[15272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:27:21.509686 15810 tablet_replica.cc:330] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15: stopping tablet replica
I20260618 22:27:21.514638 15812 ts_tablet_manager.cc:1918] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:27:21.514834 15812 log.cc:1196] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/eda727675b0748eb8d0c131a5c09a3ff
I20260618 22:27:21.515204 15785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.450104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50568 (local address 127.10.242.193:39401)
0618 22:27:21.450285 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:21.450291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:21.487033 (+ 36742us) server_negotiation.cc:407] Connection header received
0618 22:27:21.487396 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:21.487403 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:21.487492 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:21.487580 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:21.497260 (+ 9680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.498522 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.500718 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.501482 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.501591 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:21.507437 (+ 5846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:21.507460 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:21.507482 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:21.507537 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:21.511546 (+ 4009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:21.512516 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:21.512527 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:21.512531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:21.512721 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:21.513205 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:21.513211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:21.513214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:21.513697 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:21.513981 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:21.514554 (+ 573us) server_negotiation.cc:299] Negotiation successful
0618 22:27:21.514809 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260618 22:27:21.515381 15812 ts_tablet_manager.cc:1939] T eda727675b0748eb8d0c131a5c09a3ff P cc09c571c1ad4057bb0b8e5a64dd1c2d: Deleting consensus metadata
I20260618 22:27:21.515793 15810 raft_consensus.cc:2229] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:21.516552 15810 raft_consensus.cc:2258] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:21.516691 15499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:21.517307 15300 catalog_manager.cc:4841] TS cc09c571c1ad4057bb0b8e5a64dd1c2d (127.10.242.194:37691): tablet eda727675b0748eb8d0c131a5c09a3ff (table test-table [id=22077ec6aaa548988400b780e748bb41]) successfully deleted
I20260618 22:27:21.520030 15811 ts_tablet_manager.cc:1918] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:27:21.520252 15811 log.cc:1196] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/eda727675b0748eb8d0c131a5c09a3ff
I20260618 22:27:21.520735 15811 ts_tablet_manager.cc:1939] T eda727675b0748eb8d0c131a5c09a3ff P 127d235663484d089e619d73b8d7e2e3: Deleting consensus metadata
I20260618 22:27:21.522523 15303 catalog_manager.cc:4841] TS 127d235663484d089e619d73b8d7e2e3 (127.10.242.195:38709): tablet eda727675b0748eb8d0c131a5c09a3ff (table test-table [id=22077ec6aaa548988400b780e748bb41]) successfully deleted
I20260618 22:27:21.607271 15810 ts_tablet_manager.cc:1905] T eda727675b0748eb8d0c131a5c09a3ff P fb758d2f58e84da3b598515ed0989b15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:21.609272 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 15377
I20260618 22:27:21.622941 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 15510
W20260618 22:27:21.622934 15300 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.242.193:39401 (error 108)
I20260618 22:27:21.625388 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.624527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39401 (local address 127.0.0.1:50580)
0618 22:27:21.624957 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:21.625151 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:39401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":208,"threads_started":1}
W20260618 22:27:21.625722 15300 catalog_manager.cc:4584] TS fb758d2f58e84da3b598515ed0989b15 (127.10.242.193:39401): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eda727675b0748eb8d0c131a5c09a3ff: Network error: Client connection negotiation failed: client connection to 127.10.242.193:39401: connect: Connection refused (error 111)
Jun 18 22:27:21 dist-test-slave-0956 krb5kdc[15272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.194@KRBTEST.COM
I20260618 22:27:21.640256 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.623815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58412 (local address 127.10.242.194:37691)
0618 22:27:21.623995 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:21.624001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:21.624128 (+ 127us) server_negotiation.cc:407] Connection header received
0618 22:27:21.624340 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:21.624345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:21.624410 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:21.624491 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:21.626052 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.626789 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.628648 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.629318 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.629434 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:21.634584 (+ 5150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:21.634601 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:21.634617 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:21.634666 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:21.637503 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:21.638349 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:21.638358 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:21.638360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:21.638488 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:21.638866 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:21.638869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:21.638871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:21.639270 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:21.639473 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:21.639794 (+ 321us) server_negotiation.cc:299] Negotiation successful
0618 22:27:21.639949 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:27:21.640882 15632 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:21.679412 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.678896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39401 (local address 127.0.0.1:50588)
0618 22:27:21.679127 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:21.679244 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:39401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260618 22:27:21.680572 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.680154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39401 (local address 127.0.0.1:50598)
0618 22:27:21.680311 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:21.680400 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:39401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260618 22:27:21.730093 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 15510
I20260618 22:27:21.741530 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.740984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39401 (local address 127.0.0.1:50600)
0618 22:27:21.741254 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:21.741359 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:39401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260618 22:27:21.742736 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.742302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39401 (local address 127.0.0.1:50602)
0618 22:27:21.742465 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:21.742560 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:39401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260618 22:27:21.743259 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 15642
Jun 18 22:27:21 dist-test-slave-0956 krb5kdc[15272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.195@KRBTEST.COM
I20260618 22:27:21.760071 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.743910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49866 (local address 127.10.242.195:38709)
0618 22:27:21.744095 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:21.744099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:21.744266 (+ 167us) server_negotiation.cc:407] Connection header received
0618 22:27:21.744405 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:21.744414 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:21.744505 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:21.744617 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:21.745853 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.747059 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.748333 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:21.749034 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:21.749211 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:21.753836 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:21.753853 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:21.753871 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:21.753917 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:21.757034 (+ 3117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:21.758034 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:21.758045 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:21.758047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:21.758209 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:21.758653 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:21.758657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:21.758659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:21.759020 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:21.759264 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:21.759584 (+ 320us) server_negotiation.cc:299] Negotiation successful
0618 22:27:21.759759 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260618 22:27:21.760737 15764 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:21.823463 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.822991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39401 (local address 127.0.0.1:50618)
0618 22:27:21.823211 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:21.823307 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:39401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260618 22:27:21.824457 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:21.824065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39401 (local address 127.0.0.1:50624)
0618 22:27:21.824241 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:21.824308 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:39401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260618 22:27:21.882798 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 15642
I20260618 22:27:21.897339 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 15285
I20260618 22:27:21.898494 15346 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:22.003005 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 15285
2026-06-18T22:27:22Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4166 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-18T22:27:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:27:22Z Disabled control of system clock
I20260618 22:27:22.063561 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:44853
--webserver_interface=127.10.242.254
--webserver_port=0
--builtin_ntp_servers=127.10.242.212:38655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:44853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260618 22:27:22.375731 15823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:22.376025 15823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:22.376134 15823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:22.383440 15823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:27:22.383556 15823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:22.383600 15823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:22.383680 15823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:27:22.383738 15823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:27:22.392917 15823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:38655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:44853
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:44853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:22.394251 15823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:22.396039 15823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:22.407688 15828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:22.407765 15829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:22.407816 15831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:22.408334 15823 server_base.cc:1034] running on GCE node
I20260618 22:27:22.409413 15823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:22.410981 15823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:22.412249 15823 hybrid_clock.cc:648] HybridClock initialized: now 1781821642412180 us; error 61 us; skew 500 ppm
I20260618 22:27:22.412597 15823 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:27:22.415035 15823 webserver.cc:460] Webserver started at http://127.10.242.254:44625/ using document root <none> and password file <none>
I20260618 22:27:22.415741 15823 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:22.415843 15823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:22.416141 15823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:22.418984 15823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "f0dc91ef5dfb472c9ce3175db2e056b5"
format_stamp: "Formatted at 2026-06-18 22:27:22 on dist-test-slave-0956"
server_key: "74b8f88a8fa06454f80b44d44d4873a8"
server_key_iv: "ec49c4850d19d1f2db253ed196b7892a"
server_key_version: "encryptionkey@0"
I20260618 22:27:22.419747 15823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "f0dc91ef5dfb472c9ce3175db2e056b5"
format_stamp: "Formatted at 2026-06-18 22:27:22 on dist-test-slave-0956"
server_key: "74b8f88a8fa06454f80b44d44d4873a8"
server_key_iv: "ec49c4850d19d1f2db253ed196b7892a"
server_key_version: "encryptionkey@0"
I20260618 22:27:22.424712 15823 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260618 22:27:22.428138 15837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:22.429661 15823 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:27:22.429821 15823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "f0dc91ef5dfb472c9ce3175db2e056b5"
format_stamp: "Formatted at 2026-06-18 22:27:22 on dist-test-slave-0956"
server_key: "74b8f88a8fa06454f80b44d44d4873a8"
server_key_iv: "ec49c4850d19d1f2db253ed196b7892a"
server_key_version: "encryptionkey@0"
I20260618 22:27:22.430015 15823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:22.435499 15823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:22.436259 15823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:22.436527 15823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:22.453112 15823 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:44853
I20260618 22:27:22.453200 15888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:44853 every 8 connection(s)
I20260618 22:27:22.454501 15823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:27:22.458256 15889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:22.461022 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 15823
I20260618 22:27:22.461275 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:27:22.462119 11211 external_mini_cluster.cc:1428] Setting key 5e92d2a0a58a4e7ed2216efe67625982
I20260618 22:27:22.470821 15892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:22.464548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48484 (local address 127.10.242.254:44853)
0618 22:27:22.465594 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:22.465606 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:22.465658 (+ 52us) server_negotiation.cc:407] Connection header received
0618 22:27:22.467311 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:22.467334 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:22.467806 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:22.468185 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:22.469251 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:22.469257 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:22.469272 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:22.469285 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:22.469348 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:22.469364 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:22.469558 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:22.469702 (+ 144us) server_negotiation.cc:299] Negotiation successful
0618 22:27:22.469939 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":364,"threads_started":1}
I20260618 22:27:22.470930 15889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5: Bootstrap starting.
I20260618 22:27:22.474375 15889 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:22.475440 15889 log.cc:824] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:22.478232 15889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5: No bootstrap required, opened a new log
I20260618 22:27:22.485437 15889 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0dc91ef5dfb472c9ce3175db2e056b5" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44853 } }
I20260618 22:27:22.485793 15889 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:22.485867 15889 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0dc91ef5dfb472c9ce3175db2e056b5, State: Initialized, Role: FOLLOWER
I20260618 22:27:22.486363 15889 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0dc91ef5dfb472c9ce3175db2e056b5" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44853 } }
I20260618 22:27:22.486546 15889 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:27:22.486637 15889 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:27:22.486761 15889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:22.488945 15889 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0dc91ef5dfb472c9ce3175db2e056b5" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44853 } }
I20260618 22:27:22.489499 15889 leader_election.cc:302] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0dc91ef5dfb472c9ce3175db2e056b5; no voters:
I20260618 22:27:22.490049 15889 leader_election.cc:288] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:27:22.490173 15894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:22.490779 15894 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5 [term 1 LEADER]: Becoming Leader. State: Replica: f0dc91ef5dfb472c9ce3175db2e056b5, State: Running, Role: LEADER
I20260618 22:27:22.491369 15894 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0dc91ef5dfb472c9ce3175db2e056b5" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44853 } }
I20260618 22:27:22.491963 15889 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:27:22.493839 15895 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0dc91ef5dfb472c9ce3175db2e056b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0dc91ef5dfb472c9ce3175db2e056b5" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44853 } } }
I20260618 22:27:22.494215 15895 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:22.494848 15896 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0dc91ef5dfb472c9ce3175db2e056b5. Latest consensus state: current_term: 1 leader_uuid: "f0dc91ef5dfb472c9ce3175db2e056b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0dc91ef5dfb472c9ce3175db2e056b5" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44853 } } }
I20260618 22:27:22.495245 15896 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:22.496178 15903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:27:22.501173 15903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:27:22.509598 15903 catalog_manager.cc:1347] Generated new cluster ID: a6dc9b61e40e4e6fa1cb395a7a98d3de
I20260618 22:27:22.509706 15903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:27:22.535673 15903 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:27:22.536146 15903 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:27:22.550052 15903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5: Generated new TSK 0
I20260618 22:27:22.551014 15903 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:27:22.556633 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:44853
--builtin_ntp_servers=127.10.242.212:38655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260618 22:27:22.899569 15913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:22.899845 15913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:22.899930 15913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:22.906981 15913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:22.907083 15913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:22.907191 15913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:27:22.916157 15913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:38655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:44853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:22.917412 15913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:22.919027 15913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:22.931893 15918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:22.931839 15919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:22.931861 15921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:22.932693 15913 server_base.cc:1034] running on GCE node
I20260618 22:27:22.933367 15913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:22.934302 15913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:22.935518 15913 hybrid_clock.cc:648] HybridClock initialized: now 1781821642935453 us; error 55 us; skew 500 ppm
I20260618 22:27:22.935837 15913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:27:22.938361 15913 webserver.cc:460] Webserver started at http://127.10.242.193:33565/ using document root <none> and password file <none>
I20260618 22:27:22.939118 15913 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:22.939213 15913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:22.939533 15913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:22.942616 15913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "25d91875c01348c890ecf9c8d1905e35"
format_stamp: "Formatted at 2026-06-18 22:27:22 on dist-test-slave-0956"
server_key: "9359a031af1d839cfec00fa97d6950a9"
server_key_iv: "c4d461f5196dbcb0187a50cf83465d5b"
server_key_version: "encryptionkey@0"
I20260618 22:27:22.943389 15913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "25d91875c01348c890ecf9c8d1905e35"
format_stamp: "Formatted at 2026-06-18 22:27:22 on dist-test-slave-0956"
server_key: "9359a031af1d839cfec00fa97d6950a9"
server_key_iv: "c4d461f5196dbcb0187a50cf83465d5b"
server_key_version: "encryptionkey@0"
I20260618 22:27:22.948575 15913 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260618 22:27:22.952068 15927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:22.953477 15913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:27:22.953619 15913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "25d91875c01348c890ecf9c8d1905e35"
format_stamp: "Formatted at 2026-06-18 22:27:22 on dist-test-slave-0956"
server_key: "9359a031af1d839cfec00fa97d6950a9"
server_key_iv: "c4d461f5196dbcb0187a50cf83465d5b"
server_key_version: "encryptionkey@0"
I20260618 22:27:22.953832 15913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:22.959306 15913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:22.960069 15913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:22.960311 15913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:22.961263 15913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:22.962608 15913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:22.962692 15913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:22.962776 15913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:22.962828 15913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:22.989763 15913 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:36531
I20260618 22:27:22.989809 16039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:36531 every 8 connection(s)
I20260618 22:27:22.991005 15913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:27:22.997556 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 15913
I20260618 22:27:22.997758 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:27:22.998435 11211 external_mini_cluster.cc:1428] Setting key b9738a1b8537a9b6d4ea258357437a83
I20260618 22:27:22.999125 16042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:22.993460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:52855 (local address 127.10.242.254:44853)
0618 22:27:22.993842 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:22.993847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:22.995111 (+ 1264us) server_negotiation.cc:407] Connection header received
0618 22:27:22.996860 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:22.996865 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:22.996933 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:22.997027 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:22.998412 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:22.998418 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:22.998423 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:22.998443 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:22.998482 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:22.998494 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:22.998591 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:22.998823 (+ 232us) server_negotiation.cc:299] Negotiation successful
0618 22:27:22.998870 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":191,"threads_started":1}
I20260618 22:27:23.000115 16043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:22.994002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:44853 (local address 127.10.242.193:52855)
0618 22:27:22.994945 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:22.994986 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:22.996588 (+ 1602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:22.997330 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:22.997339 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:22.997438 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:22.998107 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:27:22.998110 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:22.998150 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:27:22.998237 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:22.998600 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:22.998613 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:22.998622 (+ 9us) client_negotiation.cc:769] Sending connection context
0618 22:27:22.998915 (+ 293us) client_negotiation.cc:240] Negotiation successful
0618 22:27:22.999169 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":241,"threads_started":1}
I20260618 22:27:23.001304 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:44853
--builtin_ntp_servers=127.10.242.212:38655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260618 22:27:23.001578 16040 heartbeater.cc:347] Connected to a master server at 127.10.242.254:44853
I20260618 22:27:23.001896 16040 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:23.002463 16040 heartbeater.cc:511] Master 127.10.242.254:44853 requested a full tablet report, sending...
I20260618 22:27:23.004421 15854 ts_manager.cc:194] Registered new tserver with Master: 25d91875c01348c890ecf9c8d1905e35 (127.10.242.193:36531)
W20260618 22:27:23.317341 16044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:23.317641 16044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:23.317755 16044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:23.324935 16044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:23.325054 16044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:23.325291 16044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:27:23.334512 16044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:38655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:44853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:23.335762 16044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:23.337559 16044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:23.350028 16049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:23.349999 16050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:23.349973 16052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:23.351046 16044 server_base.cc:1034] running on GCE node
I20260618 22:27:23.351683 16044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:23.352596 16044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:23.353802 16044 hybrid_clock.cc:648] HybridClock initialized: now 1781821643353743 us; error 50 us; skew 500 ppm
I20260618 22:27:23.354111 16044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:27:23.356271 16044 webserver.cc:460] Webserver started at http://127.10.242.194:37839/ using document root <none> and password file <none>
I20260618 22:27:23.356963 16044 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:23.357057 16044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:23.357401 16044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:23.360105 16044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "a63daae66ff141b19d2311bd9704901d"
format_stamp: "Formatted at 2026-06-18 22:27:23 on dist-test-slave-0956"
server_key: "8e67a9b089f4e1e6b2b27414a0718f70"
server_key_iv: "d37733a5102d2750b7515eb620e9c303"
server_key_version: "encryptionkey@0"
I20260618 22:27:23.360843 16044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "a63daae66ff141b19d2311bd9704901d"
format_stamp: "Formatted at 2026-06-18 22:27:23 on dist-test-slave-0956"
server_key: "8e67a9b089f4e1e6b2b27414a0718f70"
server_key_iv: "d37733a5102d2750b7515eb620e9c303"
server_key_version: "encryptionkey@0"
I20260618 22:27:23.365818 16044 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260618 22:27:23.369216 16058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:23.370640 16044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:27:23.370795 16044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "a63daae66ff141b19d2311bd9704901d"
format_stamp: "Formatted at 2026-06-18 22:27:23 on dist-test-slave-0956"
server_key: "8e67a9b089f4e1e6b2b27414a0718f70"
server_key_iv: "d37733a5102d2750b7515eb620e9c303"
server_key_version: "encryptionkey@0"
I20260618 22:27:23.370994 16044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:23.376461 16044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:23.377254 16044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:23.377492 16044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:23.378345 16044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:23.379648 16044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:23.379742 16044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:23.379846 16044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:23.379911 16044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:23.409027 16044 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:44939
I20260618 22:27:23.409102 16170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:44939 every 8 connection(s)
I20260618 22:27:23.410334 16044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:27:23.418318 16042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:23.412774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:45055 (local address 127.10.242.254:44853)
0618 22:27:23.412969 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:23.412975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:23.414404 (+ 1429us) server_negotiation.cc:407] Connection header received
0618 22:27:23.416066 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:23.416070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:23.416195 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:23.416334 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:23.417681 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:23.417686 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:23.417689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:23.417707 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:23.417741 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:23.417754 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:23.417861 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:23.418006 (+ 145us) server_negotiation.cc:299] Negotiation successful
0618 22:27:23.418061 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260618 22:27:23.419196 16173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:23.413394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:44853 (local address 127.10.242.194:45055)
0618 22:27:23.414245 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:23.414290 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:23.415746 (+ 1456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:23.416556 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:23.416574 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:23.416700 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:23.417375 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:27:23.417378 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:23.417417 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:27:23.417503 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:23.417834 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:23.417839 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:23.417844 (+ 5us) client_negotiation.cc:769] Sending connection context
0618 22:27:23.418044 (+ 200us) client_negotiation.cc:240] Negotiation successful
0618 22:27:23.418258 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":240,"threads_started":1}
I20260618 22:27:23.419689 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 16044
I20260618 22:27:23.419893 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:27:23.420384 11211 external_mini_cluster.cc:1428] Setting key a44d839aa3decbcc98985e3e8a5ba55a
I20260618 22:27:23.420701 16171 heartbeater.cc:347] Connected to a master server at 127.10.242.254:44853
I20260618 22:27:23.420984 16171 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:23.421630 16171 heartbeater.cc:511] Master 127.10.242.254:44853 requested a full tablet report, sending...
I20260618 22:27:23.422989 15854 ts_manager.cc:194] Registered new tserver with Master: a63daae66ff141b19d2311bd9704901d (127.10.242.194:44939)
I20260618 22:27:23.423235 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:44853
--builtin_ntp_servers=127.10.242.212:38655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260618 22:27:23.750244 16174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:23.750524 16174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:23.750612 16174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:23.757872 16174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:23.757997 16174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:23.758118 16174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:27:23.767843 16174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:38655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:44853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:23.769150 16174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:23.770967 16174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:23.784355 16182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:23.784441 16179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:23.784391 16180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:23.786423 16174 server_base.cc:1034] running on GCE node
I20260618 22:27:23.786998 16174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:23.787953 16174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:23.789167 16174 hybrid_clock.cc:648] HybridClock initialized: now 1781821643789111 us; error 46 us; skew 500 ppm
I20260618 22:27:23.789475 16174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:27:23.791697 16174 webserver.cc:460] Webserver started at http://127.10.242.195:38527/ using document root <none> and password file <none>
I20260618 22:27:23.792412 16174 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:23.792511 16174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:23.792820 16174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:23.795766 16174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "054b3f954ea14691b96b1b7d994b23ca"
format_stamp: "Formatted at 2026-06-18 22:27:23 on dist-test-slave-0956"
server_key: "1886045d7654ae801703fbe28e5d02b3"
server_key_iv: "a516287bd51a3f8bc1e02d6a2ef270f4"
server_key_version: "encryptionkey@0"
I20260618 22:27:23.796540 16174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "054b3f954ea14691b96b1b7d994b23ca"
format_stamp: "Formatted at 2026-06-18 22:27:23 on dist-test-slave-0956"
server_key: "1886045d7654ae801703fbe28e5d02b3"
server_key_iv: "a516287bd51a3f8bc1e02d6a2ef270f4"
server_key_version: "encryptionkey@0"
I20260618 22:27:23.801668 16174 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260618 22:27:23.805133 16188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:23.806597 16174 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:27:23.806767 16174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "054b3f954ea14691b96b1b7d994b23ca"
format_stamp: "Formatted at 2026-06-18 22:27:23 on dist-test-slave-0956"
server_key: "1886045d7654ae801703fbe28e5d02b3"
server_key_iv: "a516287bd51a3f8bc1e02d6a2ef270f4"
server_key_version: "encryptionkey@0"
I20260618 22:27:23.806981 16174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:23.812621 16174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:23.813439 16174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:23.813679 16174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:23.814555 16174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:23.815994 16174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:23.816102 16174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:23.816203 16174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:23.816279 16174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:23.846336 16174 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:45575
I20260618 22:27:23.846395 16300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:45575 every 8 connection(s)
I20260618 22:27:23.847633 16174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:27:23.851284 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 16174
I20260618 22:27:23.851467 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:27:23.852002 11211 external_mini_cluster.cc:1428] Setting key 32ac2e775c7e84aa3d29d1c8a4772899
I20260618 22:27:23.856384 16042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:23.850115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:44459 (local address 127.10.242.254:44853)
0618 22:27:23.850314 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:23.850320 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:23.851991 (+ 1671us) server_negotiation.cc:407] Connection header received
0618 22:27:23.854139 (+ 2148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:23.854145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:23.854268 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:23.854380 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:23.855695 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:23.855700 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:23.855705 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:23.855722 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:23.855751 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:23.855762 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:23.855865 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:23.856036 (+ 171us) server_negotiation.cc:299] Negotiation successful
0618 22:27:23.856112 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260618 22:27:23.857385 16303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:23.850731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:44853 (local address 127.10.242.195:44459)
0618 22:27:23.851809 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:23.851859 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:23.853831 (+ 1972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:23.854592 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:23.854600 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:23.854695 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:23.855406 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:27:23.855409 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:23.855445 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:27:23.855529 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:23.855846 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:23.855856 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:23.855864 (+ 8us) client_negotiation.cc:769] Sending connection context
0618 22:27:23.856116 (+ 252us) client_negotiation.cc:240] Negotiation successful
0618 22:27:23.856378 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":283,"threads_started":1}
I20260618 22:27:23.858644 16301 heartbeater.cc:347] Connected to a master server at 127.10.242.254:44853
I20260618 22:27:23.858953 16301 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:23.859571 16301 heartbeater.cc:511] Master 127.10.242.254:44853 requested a full tablet report, sending...
I20260618 22:27:23.860833 15854 ts_manager.cc:194] Registered new tserver with Master: 054b3f954ea14691b96b1b7d994b23ca (127.10.242.195:45575)
I20260618 22:27:23.867542 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:27:23.874577 16042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:23.871995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48486 (local address 127.10.242.254:44853)
0618 22:27:23.872238 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:23.872243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:23.872706 (+ 463us) server_negotiation.cc:407] Connection header received
0618 22:27:23.872881 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:23.872885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:23.872947 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:23.873045 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:23.873963 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:23.873967 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:23.873970 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:23.873990 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:23.874026 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:23.874040 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:23.874151 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:23.874306 (+ 155us) server_negotiation.cc:299] Negotiation successful
0618 22:27:23.874357 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260618 22:27:23.878667 15854 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48486:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:27:23.885233 15854 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:27:23.902091 16318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:23.898648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57270 (local address 127.10.242.195:45575)
0618 22:27:23.899839 (+ 1191us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:23.899849 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:23.899880 (+ 31us) server_negotiation.cc:407] Connection header received
0618 22:27:23.899947 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:23.899953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:23.900139 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:23.900289 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:23.901258 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:23.901263 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:23.901274 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:23.901286 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:23.901324 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:23.901337 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:23.901478 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:23.901632 (+ 154us) server_negotiation.cc:299] Negotiation successful
0618 22:27:23.901732 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1017,"thread_start_us":1627,"threads_started":1}
I20260618 22:27:23.902173 16317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:23.898136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:45575 (local address 127.0.0.1:57270)
0618 22:27:23.899651 (+ 1515us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:23.899666 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:23.899756 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:23.900368 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:23.900373 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:23.900394 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:23.900981 (+ 587us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:27:23.900984 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:23.900999 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:27:23.901042 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:23.901423 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:23.901433 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:23.901441 (+ 8us) client_negotiation.cc:769] Sending connection context
0618 22:27:23.901602 (+ 161us) client_negotiation.cc:240] Negotiation successful
0618 22:27:23.901679 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1411,"thread_start_us":1291,"threads_started":1}
I20260618 22:27:23.902356 16313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:23.896423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:44939 (local address 127.0.0.1:46908)
0618 22:27:23.897290 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:23.897320 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:23.897450 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:23.898362 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:23.898370 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:23.898467 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:23.899234 (+ 767us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:27:23.899238 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:23.899285 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:27:23.899373 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:23.901788 (+ 2415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:23.901792 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:23.901797 (+ 5us) client_negotiation.cc:769] Sending connection context
0618 22:27:23.901908 (+ 111us) client_negotiation.cc:240] Negotiation successful
0618 22:27:23.901954 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":333,"threads_started":1}
I20260618 22:27:23.902822 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:23.897848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44372 (local address 127.10.242.193:36531)
0618 22:27:23.900179 (+ 2331us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:23.900187 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:23.900212 (+ 25us) server_negotiation.cc:407] Connection header received
0618 22:27:23.900272 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:23.900279 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:23.900472 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:23.900602 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:23.901797 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:23.901804 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:23.901815 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:23.901850 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:23.901896 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:23.901914 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:23.902402 (+ 488us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:23.902467 (+ 65us) server_negotiation.cc:299] Negotiation successful
0618 22:27:23.902546 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2185,"thread_start_us":2077,"threads_started":1}
I20260618 22:27:23.903318 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:23.897578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:36531 (local address 127.0.0.1:44372)
0618 22:27:23.898068 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:23.898086 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:23.898183 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:23.900620 (+ 2437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:23.900624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:23.900646 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:23.901162 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:27:23.901166 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:23.901180 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:27:23.901230 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:23.902041 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:23.902049 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:23.902055 (+ 6us) client_negotiation.cc:769] Sending connection context
0618 22:27:23.902188 (+ 133us) client_negotiation.cc:240] Negotiation successful
0618 22:27:23.902246 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":233,"threads_started":1}
I20260618 22:27:23.903672 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:23.896584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46908 (local address 127.10.242.194:44939)
0618 22:27:23.897215 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:23.897224 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:23.897415 (+ 191us) server_negotiation.cc:407] Connection header received
0618 22:27:23.897595 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:23.897604 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:23.897885 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:23.898032 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:23.901563 (+ 3531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:23.901571 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:23.901585 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:23.901609 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:23.901662 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:23.901680 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:23.902486 (+ 806us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:23.902555 (+ 69us) server_negotiation.cc:299] Negotiation successful
0618 22:27:23.902647 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":336,"threads_started":1}
I20260618 22:27:23.906412 15975 tablet_service.cc:1467] Processing CreateTablet for tablet 8d22ea4c51064e149909f0a67380c411 (DEFAULT_TABLE table=test-table [id=5b876581ebb047c7846c429f440aae5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:23.906617 16236 tablet_service.cc:1467] Processing CreateTablet for tablet 8d22ea4c51064e149909f0a67380c411 (DEFAULT_TABLE table=test-table [id=5b876581ebb047c7846c429f440aae5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:23.906688 16106 tablet_service.cc:1467] Processing CreateTablet for tablet 8d22ea4c51064e149909f0a67380c411 (DEFAULT_TABLE table=test-table [id=5b876581ebb047c7846c429f440aae5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:23.907706 15975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d22ea4c51064e149909f0a67380c411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:23.907982 16236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d22ea4c51064e149909f0a67380c411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:23.907984 16106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d22ea4c51064e149909f0a67380c411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:23.919430 16319 tablet_bootstrap.cc:492] T 8d22ea4c51064e149909f0a67380c411 P 25d91875c01348c890ecf9c8d1905e35: Bootstrap starting.
I20260618 22:27:23.921902 16321 tablet_bootstrap.cc:492] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca: Bootstrap starting.
I20260618 22:27:23.923590 16319 tablet_bootstrap.cc:654] T 8d22ea4c51064e149909f0a67380c411 P 25d91875c01348c890ecf9c8d1905e35: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:23.924882 16319 log.cc:824] T 8d22ea4c51064e149909f0a67380c411 P 25d91875c01348c890ecf9c8d1905e35: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:23.925586 16321 tablet_bootstrap.cc:654] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:23.926213 16320 tablet_bootstrap.cc:492] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d: Bootstrap starting.
I20260618 22:27:23.926894 16321 log.cc:824] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:23.928267 16319 tablet_bootstrap.cc:492] T 8d22ea4c51064e149909f0a67380c411 P 25d91875c01348c890ecf9c8d1905e35: No bootstrap required, opened a new log
I20260618 22:27:23.928505 16319 ts_tablet_manager.cc:1397] T 8d22ea4c51064e149909f0a67380c411 P 25d91875c01348c890ecf9c8d1905e35: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260618 22:27:23.929476 16320 tablet_bootstrap.cc:654] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:23.929931 16321 tablet_bootstrap.cc:492] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca: No bootstrap required, opened a new log
I20260618 22:27:23.930259 16321 ts_tablet_manager.cc:1397] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260618 22:27:23.930606 16320 log.cc:824] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:23.933624 16320 tablet_bootstrap.cc:492] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d: No bootstrap required, opened a new log
I20260618 22:27:23.933895 16320 ts_tablet_manager.cc:1397] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260618 22:27:23.936122 16319 raft_consensus.cc:348] T 8d22ea4c51064e149909f0a67380c411 P 25d91875c01348c890ecf9c8d1905e35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a63daae66ff141b19d2311bd9704901d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44939 } } peers { permanent_uuid: "25d91875c01348c890ecf9c8d1905e35" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36531 } } peers { permanent_uuid: "054b3f954ea14691b96b1b7d994b23ca" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 45575 } }
I20260618 22:27:23.936491 16319 raft_consensus.cc:374] T 8d22ea4c51064e149909f0a67380c411 P 25d91875c01348c890ecf9c8d1905e35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:23.936576 16319 raft_consensus.cc:729] T 8d22ea4c51064e149909f0a67380c411 P 25d91875c01348c890ecf9c8d1905e35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25d91875c01348c890ecf9c8d1905e35, State: Initialized, Role: FOLLOWER
I20260618 22:27:23.937175 16319 consensus_queue.cc:260] T 8d22ea4c51064e149909f0a67380c411 P 25d91875c01348c890ecf9c8d1905e35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a63daae66ff141b19d2311bd9704901d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44939 } } peers { permanent_uuid: "25d91875c01348c890ecf9c8d1905e35" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36531 } } peers { permanent_uuid: "054b3f954ea14691b96b1b7d994b23ca" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 45575 } }
I20260618 22:27:23.940433 16321 raft_consensus.cc:348] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a63daae66ff141b19d2311bd9704901d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44939 } } peers { permanent_uuid: "25d91875c01348c890ecf9c8d1905e35" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36531 } } peers { permanent_uuid: "054b3f954ea14691b96b1b7d994b23ca" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 45575 } }
I20260618 22:27:23.940893 16321 raft_consensus.cc:374] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:23.940999 16321 raft_consensus.cc:729] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 054b3f954ea14691b96b1b7d994b23ca, State: Initialized, Role: FOLLOWER
I20260618 22:27:23.942047 16321 consensus_queue.cc:260] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a63daae66ff141b19d2311bd9704901d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44939 } } peers { permanent_uuid: "25d91875c01348c890ecf9c8d1905e35" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36531 } } peers { permanent_uuid: "054b3f954ea14691b96b1b7d994b23ca" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 45575 } }
I20260618 22:27:23.943327 16301 heartbeater.cc:503] Master 127.10.242.254:44853 was elected leader, sending a full tablet report...
I20260618 22:27:23.943975 16321 ts_tablet_manager.cc:1428] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca: Time spent starting tablet: real 0.014s user 0.010s sys 0.002s
I20260618 22:27:23.943756 16320 raft_consensus.cc:348] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a63daae66ff141b19d2311bd9704901d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44939 } } peers { permanent_uuid: "25d91875c01348c890ecf9c8d1905e35" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36531 } } peers { permanent_uuid: "054b3f954ea14691b96b1b7d994b23ca" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 45575 } }
I20260618 22:27:23.944221 16320 raft_consensus.cc:374] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:23.944306 16320 raft_consensus.cc:729] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a63daae66ff141b19d2311bd9704901d, State: Initialized, Role: FOLLOWER
I20260618 22:27:23.944875 16320 consensus_queue.cc:260] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a63daae66ff141b19d2311bd9704901d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44939 } } peers { permanent_uuid: "25d91875c01348c890ecf9c8d1905e35" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36531 } } peers { permanent_uuid: "054b3f954ea14691b96b1b7d994b23ca" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 45575 } }
I20260618 22:27:23.946329 16319 ts_tablet_manager.cc:1428] T 8d22ea4c51064e149909f0a67380c411 P 25d91875c01348c890ecf9c8d1905e35: Time spent starting tablet: real 0.018s user 0.008s sys 0.011s
I20260618 22:27:23.946723 16040 heartbeater.cc:503] Master 127.10.242.254:44853 was elected leader, sending a full tablet report...
I20260618 22:27:23.948262 16171 heartbeater.cc:503] Master 127.10.242.254:44853 was elected leader, sending a full tablet report...
I20260618 22:27:23.948938 16320 ts_tablet_manager.cc:1428] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
W20260618 22:27:23.993181 16041 tablet.cc:2369] T 8d22ea4c51064e149909f0a67380c411 P 25d91875c01348c890ecf9c8d1905e35: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:27:24.068897 16326 raft_consensus.cc:482] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:27:24.069280 16326 raft_consensus.cc:504] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a63daae66ff141b19d2311bd9704901d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44939 } } peers { permanent_uuid: "25d91875c01348c890ecf9c8d1905e35" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36531 } } peers { permanent_uuid: "054b3f954ea14691b96b1b7d994b23ca" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 45575 } }
I20260618 22:27:24.070899 16326 leader_election.cc:288] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a63daae66ff141b19d2311bd9704901d (127.10.242.194:44939), 25d91875c01348c890ecf9c8d1905e35 (127.10.242.193:36531)
I20260618 22:27:24.073849 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:24.071495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:56079 (local address 127.10.242.194:44939)
0618 22:27:24.071688 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:24.071694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:24.071765 (+ 71us) server_negotiation.cc:407] Connection header received
0618 22:27:24.071930 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:24.071935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:24.071994 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:24.072090 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:24.073233 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:24.073239 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:24.073244 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:24.073263 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:24.073302 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:24.073318 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:24.073451 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:24.073530 (+ 79us) server_negotiation.cc:299] Negotiation successful
0618 22:27:24.073600 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260618 22:27:24.073879 16303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:24.071425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:44939 (local address 127.10.242.195:56079)
0618 22:27:24.071653 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:24.071676 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:24.071812 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:24.072188 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:24.072205 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:24.072240 (+ 35us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:24.072986 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:27:24.072990 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:24.073009 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:27:24.073061 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:24.073441 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:24.073444 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:24.073448 (+ 4us) client_negotiation.cc:769] Sending connection context
0618 22:27:24.073531 (+ 83us) client_negotiation.cc:240] Negotiation successful
0618 22:27:24.073587 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260618 22:27:24.074834 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:24.071867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:36531 (local address 127.10.242.195:46147)
0618 22:27:24.072654 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:24.072670 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:24.072771 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:24.073133 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:24.073138 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:24.073163 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:24.073861 (+ 698us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:27:24.073865 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:24.073885 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:27:24.073926 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:24.074363 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:24.074368 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:24.074371 (+ 3us) client_negotiation.cc:769] Sending connection context
0618 22:27:24.074474 (+ 103us) client_negotiation.cc:240] Negotiation successful
0618 22:27:24.074522 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":462,"threads_started":1}
I20260618 22:27:24.074901 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:24.072055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:46147 (local address 127.10.242.193:36531)
0618 22:27:24.072241 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:24.072247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:24.072757 (+ 510us) server_negotiation.cc:407] Connection header received
0618 22:27:24.072871 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:24.072878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:24.072945 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:24.073059 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:24.074086 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:24.074092 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:24.074097 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:24.074113 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:24.074144 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:24.074159 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:24.074280 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:24.074519 (+ 239us) server_negotiation.cc:299] Negotiation successful
0618 22:27:24.074583 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260618 22:27:24.075011 16126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d22ea4c51064e149909f0a67380c411" candidate_uuid: "054b3f954ea14691b96b1b7d994b23ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a63daae66ff141b19d2311bd9704901d" is_pre_election: true
I20260618 22:27:24.075660 16126 raft_consensus.cc:2454] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 054b3f954ea14691b96b1b7d994b23ca in term 0.
I20260618 22:27:24.075817 15995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d22ea4c51064e149909f0a67380c411" candidate_uuid: "054b3f954ea14691b96b1b7d994b23ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25d91875c01348c890ecf9c8d1905e35" is_pre_election: true
I20260618 22:27:24.076327 15995 raft_consensus.cc:2454] T 8d22ea4c51064e149909f0a67380c411 P 25d91875c01348c890ecf9c8d1905e35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 054b3f954ea14691b96b1b7d994b23ca in term 0.
I20260618 22:27:24.076514 16190 leader_election.cc:302] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 054b3f954ea14691b96b1b7d994b23ca, a63daae66ff141b19d2311bd9704901d; no voters:
I20260618 22:27:24.076893 16326 raft_consensus.cc:2781] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:27:24.077037 16326 raft_consensus.cc:482] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:27:24.077188 16326 raft_consensus.cc:3037] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:24.079421 16326 raft_consensus.cc:504] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a63daae66ff141b19d2311bd9704901d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44939 } } peers { permanent_uuid: "25d91875c01348c890ecf9c8d1905e35" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36531 } } peers { permanent_uuid: "054b3f954ea14691b96b1b7d994b23ca" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 45575 } }
I20260618 22:27:24.080067 16326 leader_election.cc:288] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [CANDIDATE]: Term 1 election: Requested vote from peers a63daae66ff141b19d2311bd9704901d (127.10.242.194:44939), 25d91875c01348c890ecf9c8d1905e35 (127.10.242.193:36531)
I20260618 22:27:24.080530 16126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d22ea4c51064e149909f0a67380c411" candidate_uuid: "054b3f954ea14691b96b1b7d994b23ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a63daae66ff141b19d2311bd9704901d"
I20260618 22:27:24.080643 15995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d22ea4c51064e149909f0a67380c411" candidate_uuid: "054b3f954ea14691b96b1b7d994b23ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25d91875c01348c890ecf9c8d1905e35"
I20260618 22:27:24.080717 16126 raft_consensus.cc:3037] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:24.080883 15995 raft_consensus.cc:3037] T 8d22ea4c51064e149909f0a67380c411 P 25d91875c01348c890ecf9c8d1905e35 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:24.083050 15995 raft_consensus.cc:2454] T 8d22ea4c51064e149909f0a67380c411 P 25d91875c01348c890ecf9c8d1905e35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 054b3f954ea14691b96b1b7d994b23ca in term 1.
I20260618 22:27:24.083215 16126 raft_consensus.cc:2454] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 054b3f954ea14691b96b1b7d994b23ca in term 1.
I20260618 22:27:24.083442 16190 leader_election.cc:302] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 054b3f954ea14691b96b1b7d994b23ca, 25d91875c01348c890ecf9c8d1905e35; no voters:
I20260618 22:27:24.083732 16326 raft_consensus.cc:2781] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:24.084137 16326 raft_consensus.cc:686] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [term 1 LEADER]: Becoming Leader. State: Replica: 054b3f954ea14691b96b1b7d994b23ca, State: Running, Role: LEADER
I20260618 22:27:24.084664 16326 consensus_queue.cc:237] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a63daae66ff141b19d2311bd9704901d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44939 } } peers { permanent_uuid: "25d91875c01348c890ecf9c8d1905e35" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36531 } } peers { permanent_uuid: "054b3f954ea14691b96b1b7d994b23ca" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 45575 } }
I20260618 22:27:24.089182 15854 catalog_manager.cc:5495] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 054b3f954ea14691b96b1b7d994b23ca (127.10.242.195). New cstate: current_term: 1 leader_uuid: "054b3f954ea14691b96b1b7d994b23ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a63daae66ff141b19d2311bd9704901d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25d91875c01348c890ecf9c8d1905e35" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "054b3f954ea14691b96b1b7d994b23ca" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 45575 } health_report { overall_health: HEALTHY } } }
W20260618 22:27:24.099202 16302 tablet.cc:2369] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:27:24.144928 16318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:24.142984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57278 (local address 127.10.242.195:45575)
0618 22:27:24.143142 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:24.143146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:24.143220 (+ 74us) server_negotiation.cc:407] Connection header received
0618 22:27:24.143389 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:24.143393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:24.143469 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:24.143555 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:24.144415 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:24.144421 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:24.144424 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:24.144441 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:24.144477 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:24.144490 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:24.144612 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:24.144657 (+ 45us) server_negotiation.cc:299] Negotiation successful
0618 22:27:24.144713 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:27:24.153185 15995 raft_consensus.cc:1261] T 8d22ea4c51064e149909f0a67380c411 P 25d91875c01348c890ecf9c8d1905e35 [term 1 FOLLOWER]: Refusing update from remote peer 054b3f954ea14691b96b1b7d994b23ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:24.153185 16126 raft_consensus.cc:1261] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d [term 1 FOLLOWER]: Refusing update from remote peer 054b3f954ea14691b96b1b7d994b23ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:24.154142 16329 consensus_queue.cc:1035] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "25d91875c01348c890ecf9c8d1905e35" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:24.154521 16326 consensus_queue.cc:1035] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "a63daae66ff141b19d2311bd9704901d" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 22:27:24.162158 16172 tablet.cc:2369] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:27:24.181411 15852 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:24.181758 15852 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:24.185595 15852 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0dc91ef5dfb472c9ce3175db2e056b5: Sending DeleteTablet for 3 replicas of tablet 8d22ea4c51064e149909f0a67380c411
I20260618 22:27:24.186790 16106 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d22ea4c51064e149909f0a67380c411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:24 UTC) from {username='slave'} at 127.0.0.1:46908
I20260618 22:27:24.187168 16236 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d22ea4c51064e149909f0a67380c411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:24 UTC) from {username='slave'} at 127.0.0.1:57270
I20260618 22:27:24.187820 15975 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d22ea4c51064e149909f0a67380c411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:24 UTC) from {username='slave'} at 127.0.0.1:44372
I20260618 22:27:24.188110 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 15913
I20260618 22:27:24.210968 16343 tablet_replica.cc:330] T 8d22ea4c51064e149909f0a67380c411 P 25d91875c01348c890ecf9c8d1905e35: stopping tablet replica
I20260618 22:27:24.211577 16343 raft_consensus.cc:2229] T 8d22ea4c51064e149909f0a67380c411 P 25d91875c01348c890ecf9c8d1905e35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:24.212332 16342 tablet_replica.cc:330] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca: stopping tablet replica
I20260618 22:27:24.212960 16343 raft_consensus.cc:2258] T 8d22ea4c51064e149909f0a67380c411 P 25d91875c01348c890ecf9c8d1905e35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:24.213271 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:24.189529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44386 (local address 127.10.242.193:36531)
0618 22:27:24.189718 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:24.189725 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:24.210449 (+ 20724us) server_negotiation.cc:407] Connection header received
0618 22:27:24.210939 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:24.210949 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:24.211066 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:24.211143 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:24.212429 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:24.212435 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:24.212440 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:24.212461 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:24.212501 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:24.212518 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:24.212643 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:24.212888 (+ 245us) server_negotiation.cc:299] Negotiation successful
0618 22:27:24.212957 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:27:24.213649 16342 raft_consensus.cc:2229] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:27:24.214282 16035 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:24.214746 16341 tablet_replica.cc:330] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d: stopping tablet replica
I20260618 22:27:24.214812 16342 raft_consensus.cc:2258] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:24.215152 16341 raft_consensus.cc:2229] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:24.215627 16341 raft_consensus.cc:2258] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:24.216918 16342 ts_tablet_manager.cc:1905] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:24.217818 16341 ts_tablet_manager.cc:1905] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:24.223992 16342 ts_tablet_manager.cc:1918] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:27:24.224189 16342 log.cc:1196] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/8d22ea4c51064e149909f0a67380c411
I20260618 22:27:24.224380 16341 ts_tablet_manager.cc:1918] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:27:24.224584 16341 log.cc:1196] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/8d22ea4c51064e149909f0a67380c411
I20260618 22:27:24.224718 16342 ts_tablet_manager.cc:1939] T 8d22ea4c51064e149909f0a67380c411 P 054b3f954ea14691b96b1b7d994b23ca: Deleting consensus metadata
I20260618 22:27:24.225148 16341 ts_tablet_manager.cc:1939] T 8d22ea4c51064e149909f0a67380c411 P a63daae66ff141b19d2311bd9704901d: Deleting consensus metadata
I20260618 22:27:24.226593 15839 catalog_manager.cc:4841] TS 054b3f954ea14691b96b1b7d994b23ca (127.10.242.195:45575): tablet 8d22ea4c51064e149909f0a67380c411 (table test-table [id=5b876581ebb047c7846c429f440aae5f]) successfully deleted
I20260618 22:27:24.226933 15839 catalog_manager.cc:4841] TS a63daae66ff141b19d2311bd9704901d (127.10.242.194:44939): tablet 8d22ea4c51064e149909f0a67380c411 (table test-table [id=5b876581ebb047c7846c429f440aae5f]) successfully deleted
I20260618 22:27:24.312049 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 15913
I20260618 22:27:24.325565 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 16044
W20260618 22:27:24.325659 15839 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.242.193:36531 (error 108)
I20260618 22:27:24.327528 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:24.327067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:36531 (local address 127.0.0.1:44394)
0618 22:27:24.327240 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:24.327367 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:36531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260618 22:27:24.327766 15839 catalog_manager.cc:4584] TS 25d91875c01348c890ecf9c8d1905e35 (127.10.242.193:36531): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d22ea4c51064e149909f0a67380c411: Network error: Client connection negotiation failed: client connection to 127.10.242.193:36531: connect: Connection refused (error 111)
I20260618 22:27:24.329051 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:24.326706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46922 (local address 127.10.242.194:44939)
0618 22:27:24.326878 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:24.326885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:24.326903 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:27:24.326967 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:24.326973 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:24.327071 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:24.327171 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:24.328350 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:24.328357 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:24.328362 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:24.328383 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:24.328421 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:24.328437 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:24.328567 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:24.328695 (+ 128us) server_negotiation.cc:299] Negotiation successful
0618 22:27:24.328754 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:27:24.329726 16166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:24.381556 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:24.380995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:36531 (local address 127.0.0.1:44408)
0618 22:27:24.381244 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:24.381357 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:36531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260618 22:27:24.382686 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:24.382290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:36531 (local address 127.0.0.1:44416)
0618 22:27:24.382455 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:24.382548 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:36531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260618 22:27:24.416934 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 16044
I20260618 22:27:24.431996 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 16174
I20260618 22:27:24.438953 16318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:24.432828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57282 (local address 127.10.242.195:45575)
0618 22:27:24.433026 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:24.433031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:24.433051 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:27:24.433160 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:24.433164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:24.433212 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:24.433303 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:24.438242 (+ 4939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:24.438247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:24.438250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:24.438272 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:24.438309 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:24.438325 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:24.438464 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:24.438588 (+ 124us) server_negotiation.cc:299] Negotiation successful
0618 22:27:24.438649 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260618 22:27:24.439708 16296 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:24.443804 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:24.443376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:36531 (local address 127.0.0.1:44430)
0618 22:27:24.443541 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:24.443642 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:36531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260618 22:27:24.444993 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:24.444548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:36531 (local address 127.0.0.1:44442)
0618 22:27:24.444714 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:24.444820 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:36531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260618 22:27:24.525638 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:24.525163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:36531 (local address 127.0.0.1:44446)
0618 22:27:24.525364 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:24.525466 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:36531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260618 22:27:24.526660 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:24.526238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:36531 (local address 127.0.0.1:44456)
0618 22:27:24.526454 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:24.526535 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:36531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260618 22:27:24.556475 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 16174
I20260618 22:27:24.571288 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 15823
I20260618 22:27:24.572458 15884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:24.675530 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 15823
2026-06-18T22:27:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2670 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260618 22:27:24.711820 11211 mini_oidc.cc:152] Starting JWKS server
I20260618 22:27:24.714502 11211 webserver.cc:460] Webserver started at https://127.0.0.1:35357/ using document root <none> and password file <none>
I20260618 22:27:24.714787 11211 mini_oidc.cc:189] Starting OIDC Discovery server
I20260618 22:27:24.715406 11211 webserver.cc:460] Webserver started at http://127.0.0.1:44347/ using document root <none> and password file <none>
2026-06-18T22:27:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:27:24Z Disabled control of system clock
I20260618 22:27:24.740547 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:36033
--webserver_interface=127.10.242.254
--webserver_port=0
--builtin_ntp_servers=127.10.242.212:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:36033
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35357/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/testchainca.cert with env {}
W20260618 22:27:25.057925 16355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:25.058216 16355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:25.058302 16355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:25.065431 16355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:27:25.065542 16355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:25.065578 16355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:25.065634 16355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:27:25.065676 16355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260618 22:27:25.065735 16355 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260618 22:27:25.065773 16355 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35357/jwks.json
I20260618 22:27:25.075805 16355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:36033
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:36033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35357/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:25.077061 16355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:25.078349 16355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:25.090145 16360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:25.089977 16361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:25.090081 16363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:25.090847 16355 server_base.cc:1034] running on GCE node
I20260618 22:27:25.091640 16355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:25.093104 16355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:25.094316 16355 hybrid_clock.cc:648] HybridClock initialized: now 1781821645094270 us; error 41 us; skew 500 ppm
I20260618 22:27:25.094657 16355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:27:25.096961 16355 webserver.cc:460] Webserver started at http://127.10.242.254:38317/ using document root <none> and password file <none>
I20260618 22:27:25.097685 16355 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:25.097790 16355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:25.098076 16355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:25.101047 16355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "d75b246afccb45a381378d41c3708c38"
format_stamp: "Formatted at 2026-06-18 22:27:25 on dist-test-slave-0956"
server_key: "953b85f5efcf24f70e7041f5e1eed43a"
server_key_iv: "34c6672d79fa3b451098e720fba53f36"
server_key_version: "encryptionkey@0"
I20260618 22:27:25.101892 16355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "d75b246afccb45a381378d41c3708c38"
format_stamp: "Formatted at 2026-06-18 22:27:25 on dist-test-slave-0956"
server_key: "953b85f5efcf24f70e7041f5e1eed43a"
server_key_iv: "34c6672d79fa3b451098e720fba53f36"
server_key_version: "encryptionkey@0"
I20260618 22:27:25.107087 16355 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:27:25.110690 16369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:25.112191 16355 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:27:25.112365 16355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "d75b246afccb45a381378d41c3708c38"
format_stamp: "Formatted at 2026-06-18 22:27:25 on dist-test-slave-0956"
server_key: "953b85f5efcf24f70e7041f5e1eed43a"
server_key_iv: "34c6672d79fa3b451098e720fba53f36"
server_key_version: "encryptionkey@0"
I20260618 22:27:25.112563 16355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:25.152480 16355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:25.153389 16355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:25.153674 16355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:25.170835 16355 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:36033
I20260618 22:27:25.170855 16420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:36033 every 8 connection(s)
I20260618 22:27:25.172197 16355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:27:25.175963 16421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:25.179076 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 16355
I20260618 22:27:25.179271 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:27:25.180039 11211 external_mini_cluster.cc:1428] Setting key bf11afdfc5e50edd245a6bdfcbc4fe10
I20260618 22:27:25.188114 16421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38: Bootstrap starting.
I20260618 22:27:25.191330 16421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:25.192413 16421 log.cc:824] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:25.193490 16424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:25.182040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59772 (local address 127.10.242.254:36033)
0618 22:27:25.183002 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:25.183013 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:25.183055 (+ 42us) server_negotiation.cc:407] Connection header received
0618 22:27:25.184726 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:25.184751 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:25.185374 (+ 623us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:25.185791 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:25.186388 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:25.187649 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:25.189005 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:25.189749 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:25.189922 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:25.191093 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:25.191101 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:25.191116 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:25.191692 (+ 576us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:25.191783 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:25.191803 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:25.191907 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:25.192144 (+ 237us) server_negotiation.cc:299] Negotiation successful
0618 22:27:25.192458 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":344,"threads_started":1}
I20260618 22:27:25.195502 16421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38: No bootstrap required, opened a new log
I20260618 22:27:25.197278 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:27:25.203075 16421 raft_consensus.cc:348] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d75b246afccb45a381378d41c3708c38" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 36033 } }
I20260618 22:27:25.203460 16421 raft_consensus.cc:374] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:25.203552 16421 raft_consensus.cc:729] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d75b246afccb45a381378d41c3708c38, State: Initialized, Role: FOLLOWER
I20260618 22:27:25.204121 16421 consensus_queue.cc:260] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d75b246afccb45a381378d41c3708c38" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 36033 } }
I20260618 22:27:25.204322 16421 raft_consensus.cc:388] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:27:25.204447 16421 raft_consensus.cc:482] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:27:25.204587 16421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:25.206838 16421 raft_consensus.cc:504] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d75b246afccb45a381378d41c3708c38" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 36033 } }
I20260618 22:27:25.207445 16421 leader_election.cc:302] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d75b246afccb45a381378d41c3708c38; no voters:
I20260618 22:27:25.208004 16421 leader_election.cc:288] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:27:25.208119 16426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:25.209149 16426 raft_consensus.cc:686] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38 [term 1 LEADER]: Becoming Leader. State: Replica: d75b246afccb45a381378d41c3708c38, State: Running, Role: LEADER
I20260618 22:27:25.209898 16426 consensus_queue.cc:237] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d75b246afccb45a381378d41c3708c38" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 36033 } }
I20260618 22:27:25.210172 16421 sys_catalog.cc:564] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:27:25.212489 16428 sys_catalog.cc:455] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d75b246afccb45a381378d41c3708c38. Latest consensus state: current_term: 1 leader_uuid: "d75b246afccb45a381378d41c3708c38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d75b246afccb45a381378d41c3708c38" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 36033 } } }
I20260618 22:27:25.212894 16428 sys_catalog.cc:458] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:25.213402 16427 sys_catalog.cc:455] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d75b246afccb45a381378d41c3708c38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d75b246afccb45a381378d41c3708c38" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 36033 } } }
I20260618 22:27:25.213675 16427 sys_catalog.cc:458] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:25.222318 16435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:27:25.227085 16435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:27:25.234975 16435 catalog_manager.cc:1347] Generated new cluster ID: 8f08a28e623140f48f841e6698c194f4
I20260618 22:27:25.235100 16435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:27:25.261816 16435 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:27:25.263171 16435 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:27:25.273722 16435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d75b246afccb45a381378d41c3708c38: Generated new TSK 0
I20260618 22:27:25.274555 16435 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260618 22:27:25.319248 16451 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260618 22:27:25.319434 16451 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260618 22:27:25.320538 16424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:25.298349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59776 (local address 127.10.242.254:36033)
0618 22:27:25.298550 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:25.298556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:25.299043 (+ 487us) server_negotiation.cc:407] Connection header received
0618 22:27:25.299380 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:25.299387 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:25.299482 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:25.299666 (+ 184us) server_negotiation.cc:226] Negotiated authn=JWT
0618 22:27:25.300206 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:25.301363 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:25.302791 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:25.303482 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:25.303614 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:25.304175 (+ 561us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0618 22:27:25.319843 (+ 15668us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35357/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0618 22:27:25.320231 (+ 388us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59776: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35357/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":63}
W20260618 22:27:25.320667 16424 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59776: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35357/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260618 22:27:25.322256 16450 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:25.298225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:36033 (local address 127.0.0.1:59776)
0618 22:27:25.298941 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:25.298959 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:25.299123 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:25.299699 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:25.299703 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:25.299713 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0618 22:27:25.300053 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:25.300064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:25.301491 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:25.301496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:25.302645 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:25.302657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:25.303672 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:25.303677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:25.303820 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:25.303887 (+ 67us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0618 22:27:25.320176 (+ 16289us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35357/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0618 22:27:25.320475 (+ 299us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.242.254:36033: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35357/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":366,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.10.242.254:36033: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35357/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260618 22:27:25.324739 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 16355
I20260618 22:27:25.325882 16416 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:25.419589 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 16355
2026-06-18T22:27:25Z chronyd exiting
I20260618 22:27:25.466318 11211 test_util.cc:175] -----------------------------------------------
I20260618 22:27:25.466442 11211 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821604371818-11211-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (759 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260618 22:27:25.470168 11211 mini_oidc.cc:152] Starting JWKS server
I20260618 22:27:25.471967 11211 webserver.cc:460] Webserver started at https://127.0.0.1:41987/ using document root <none> and password file <none>
I20260618 22:27:25.472177 11211 mini_oidc.cc:189] Starting OIDC Discovery server
I20260618 22:27:25.472779 11211 webserver.cc:460] Webserver started at http://127.0.0.1:43793/ using document root <none> and password file <none>
2026-06-18T22:27:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:27:25Z Disabled control of system clock
I20260618 22:27:25.497278 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:42195
--webserver_interface=127.10.242.254
--webserver_port=0
--builtin_ntp_servers=127.10.242.212:43465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:42195
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41987/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/testchainca.cert with env {}
W20260618 22:27:25.814284 16458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:25.814584 16458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:25.814680 16458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:25.821923 16458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:27:25.822042 16458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:25.822081 16458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:25.822132 16458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:27:25.822175 16458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260618 22:27:25.822233 16458 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260618 22:27:25.822276 16458 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41987/jwks.json
I20260618 22:27:25.832198 16458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:43465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:42195
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:42195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41987/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:25.833503 16458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:25.834719 16458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:25.847052 16463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:25.847012 16466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:25.847028 16464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:25.847886 16458 server_base.cc:1034] running on GCE node
I20260618 22:27:25.848744 16458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:25.850391 16458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:25.851611 16458 hybrid_clock.cc:648] HybridClock initialized: now 1781821645851543 us; error 63 us; skew 500 ppm
I20260618 22:27:25.851953 16458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:27:25.854444 16458 webserver.cc:460] Webserver started at http://127.10.242.254:36535/ using document root <none> and password file <none>
I20260618 22:27:25.855155 16458 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:25.855252 16458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:25.855590 16458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:25.858686 16458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "dbc62a84570a477bbbc4f8b6665a3156"
format_stamp: "Formatted at 2026-06-18 22:27:25 on dist-test-slave-0956"
server_key: "29ddd0a2e1adb86e3498727e4b78a7a7"
server_key_iv: "11e62f7c7b1c6e5e641843f43b98521a"
server_key_version: "encryptionkey@0"
I20260618 22:27:25.859493 16458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "dbc62a84570a477bbbc4f8b6665a3156"
format_stamp: "Formatted at 2026-06-18 22:27:25 on dist-test-slave-0956"
server_key: "29ddd0a2e1adb86e3498727e4b78a7a7"
server_key_iv: "11e62f7c7b1c6e5e641843f43b98521a"
server_key_version: "encryptionkey@0"
I20260618 22:27:25.865206 16458 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260618 22:27:25.869010 16472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:25.870595 16458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:27:25.870784 16458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "dbc62a84570a477bbbc4f8b6665a3156"
format_stamp: "Formatted at 2026-06-18 22:27:25 on dist-test-slave-0956"
server_key: "29ddd0a2e1adb86e3498727e4b78a7a7"
server_key_iv: "11e62f7c7b1c6e5e641843f43b98521a"
server_key_version: "encryptionkey@0"
I20260618 22:27:25.871014 16458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:25.900290 16458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:25.901218 16458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:25.901535 16458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:25.918663 16458 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:42195
I20260618 22:27:25.918666 16523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:42195 every 8 connection(s)
I20260618 22:27:25.920053 16458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:27:25.924114 16524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:25.927044 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 16458
I20260618 22:27:25.927254 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:27:25.927867 11211 external_mini_cluster.cc:1428] Setting key 03f7fa88cb8792441eb2585461528d8d
I20260618 22:27:25.936281 16524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156: Bootstrap starting.
I20260618 22:27:25.939479 16524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:25.940624 16524 log.cc:824] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:25.941640 16527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:25.929956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37746 (local address 127.10.242.254:42195)
0618 22:27:25.931012 (+ 1056us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:25.931025 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:25.931071 (+ 46us) server_negotiation.cc:407] Connection header received
0618 22:27:25.932750 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:25.932776 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:25.933376 (+ 600us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:25.933806 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:25.934414 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:25.935652 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:25.937005 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:25.937755 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:25.937926 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:25.939011 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:25.939019 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:25.939034 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:25.939596 (+ 562us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:25.939723 (+ 127us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:25.939747 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:25.939858 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:25.940071 (+ 213us) server_negotiation.cc:299] Negotiation successful
0618 22:27:25.940395 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":332,"threads_started":1}
I20260618 22:27:25.943971 16524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156: No bootstrap required, opened a new log
I20260618 22:27:25.946012 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:27:25.951982 16524 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc62a84570a477bbbc4f8b6665a3156" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 42195 } }
I20260618 22:27:25.952397 16524 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:25.952490 16524 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbc62a84570a477bbbc4f8b6665a3156, State: Initialized, Role: FOLLOWER
I20260618 22:27:25.953065 16524 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc62a84570a477bbbc4f8b6665a3156" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 42195 } }
I20260618 22:27:25.953356 16524 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:27:25.953480 16524 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:27:25.953637 16524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:25.956080 16524 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc62a84570a477bbbc4f8b6665a3156" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 42195 } }
I20260618 22:27:25.956634 16524 leader_election.cc:302] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbc62a84570a477bbbc4f8b6665a3156; no voters:
I20260618 22:27:25.957223 16524 leader_election.cc:288] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:27:25.957423 16529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:25.958029 16529 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156 [term 1 LEADER]: Becoming Leader. State: Replica: dbc62a84570a477bbbc4f8b6665a3156, State: Running, Role: LEADER
I20260618 22:27:25.958616 16529 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc62a84570a477bbbc4f8b6665a3156" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 42195 } }
I20260618 22:27:25.959343 16524 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:27:25.961339 16530 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbc62a84570a477bbbc4f8b6665a3156" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc62a84570a477bbbc4f8b6665a3156" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 42195 } } }
I20260618 22:27:25.961782 16530 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:25.961699 16531 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbc62a84570a477bbbc4f8b6665a3156. Latest consensus state: current_term: 1 leader_uuid: "dbc62a84570a477bbbc4f8b6665a3156" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc62a84570a477bbbc4f8b6665a3156" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 42195 } } }
I20260618 22:27:25.961992 16531 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:25.963915 16538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:27:25.968182 16538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:27:25.977124 16538 catalog_manager.cc:1347] Generated new cluster ID: 2876e3463b0e4b4fa73f55afb9344be9
I20260618 22:27:25.977273 16538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:27:25.994935 16538 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:27:25.996326 16538 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:27:26.005319 16538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbc62a84570a477bbbc4f8b6665a3156: Generated new TSK 0
I20260618 22:27:26.006242 16538 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260618 22:27:26.043988 16554 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260618 22:27:26.044174 16554 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260618 22:27:26.045472 16527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:26.025698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37754 (local address 127.10.242.254:42195)
0618 22:27:26.025913 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:26.025919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:26.026232 (+ 313us) server_negotiation.cc:407] Connection header received
0618 22:27:26.026484 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:26.026489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:26.026567 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:26.026707 (+ 140us) server_negotiation.cc:226] Negotiated authn=JWT
0618 22:27:26.027189 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:26.027933 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:26.029396 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:26.029867 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:26.029986 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:26.030467 (+ 481us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0618 22:27:26.044742 (+ 14275us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41987/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0618 22:27:26.045166 (+ 424us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37754: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41987/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":82}
W20260618 22:27:26.045614 16527 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37754: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41987/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260618 22:27:26.045706 16553 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:26.025489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:42195 (local address 127.0.0.1:37754)
0618 22:27:26.026112 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:26.026131 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:26.026288 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:26.026739 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:26.026744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:26.026756 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0618 22:27:26.027029 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:26.027038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:26.028069 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:26.028074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:26.029244 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:26.029265 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:26.030032 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:26.030039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:26.030179 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:26.030214 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0618 22:27:26.045014 (+ 14800us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41987/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0618 22:27:26.045311 (+ 297us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.242.254:42195: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41987/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":353,"threads_started":1}
I20260618 22:27:26.046734 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 16458
I20260618 22:27:26.047904 16519 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:26.142058 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 16458
2026-06-18T22:27:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (723 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260618 22:27:26.193567 11211 mini_oidc.cc:152] Starting JWKS server
I20260618 22:27:26.195473 11211 webserver.cc:460] Webserver started at https://127.0.0.1:37703/ using document root <none> and password file <none>
I20260618 22:27:26.195701 11211 mini_oidc.cc:189] Starting OIDC Discovery server
I20260618 22:27:26.196372 11211 webserver.cc:460] Webserver started at http://127.0.0.1:42715/ using document root <none> and password file <none>
2026-06-18T22:27:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:27:26Z Disabled control of system clock
I20260618 22:27:26.221293 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:37949
--webserver_interface=127.10.242.254
--webserver_port=0
--builtin_ntp_servers=127.10.242.212:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:37949
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37703/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260618 22:27:26.538805 16561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:26.539104 16561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:26.539189 16561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:26.546419 16561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:27:26.546543 16561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:26.546579 16561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:26.546633 16561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:27:26.546675 16561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260618 22:27:26.546733 16561 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260618 22:27:26.546770 16561 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37703/jwks.json
I20260618 22:27:26.555972 16561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:37949
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:37949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37703/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:26.557255 16561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:26.559088 16561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:26.570504 16566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:26.570493 16567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:26.570531 16569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:26.571781 16561 server_base.cc:1034] running on GCE node
I20260618 22:27:26.572546 16561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:26.574043 16561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:26.575254 16561 hybrid_clock.cc:648] HybridClock initialized: now 1781821646575214 us; error 40 us; skew 500 ppm
I20260618 22:27:26.575598 16561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:27:26.577939 16561 webserver.cc:460] Webserver started at http://127.10.242.254:36085/ using document root <none> and password file <none>
I20260618 22:27:26.578622 16561 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:26.578716 16561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:26.579023 16561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:26.581928 16561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "b756a371f1d24b0da3afb630ba2b19b4"
format_stamp: "Formatted at 2026-06-18 22:27:26 on dist-test-slave-0956"
server_key: "625910ebfab800a82e64c36418acea20"
server_key_iv: "3af0758ef48d246c917e10594a770068"
server_key_version: "encryptionkey@0"
I20260618 22:27:26.582715 16561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "b756a371f1d24b0da3afb630ba2b19b4"
format_stamp: "Formatted at 2026-06-18 22:27:26 on dist-test-slave-0956"
server_key: "625910ebfab800a82e64c36418acea20"
server_key_iv: "3af0758ef48d246c917e10594a770068"
server_key_version: "encryptionkey@0"
I20260618 22:27:26.587884 16561 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:27:26.591406 16575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:26.592798 16561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260618 22:27:26.592953 16561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "b756a371f1d24b0da3afb630ba2b19b4"
format_stamp: "Formatted at 2026-06-18 22:27:26 on dist-test-slave-0956"
server_key: "625910ebfab800a82e64c36418acea20"
server_key_iv: "3af0758ef48d246c917e10594a770068"
server_key_version: "encryptionkey@0"
I20260618 22:27:26.593210 16561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:26.612023 16561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:26.612867 16561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:26.613175 16561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:26.630633 16561 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:37949
I20260618 22:27:26.630656 16626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:37949 every 8 connection(s)
I20260618 22:27:26.631989 16561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:27:26.635737 16627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:26.639710 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 16561
I20260618 22:27:26.639905 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:27:26.640674 11211 external_mini_cluster.cc:1428] Setting key 48733ac1d0922a82044ee94e3286c00a
I20260618 22:27:26.647374 16627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4: Bootstrap starting.
I20260618 22:27:26.650470 16627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:26.651494 16627 log.cc:824] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:26.654819 16627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4: No bootstrap required, opened a new log
I20260618 22:27:26.654917 16630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:26.642695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42666 (local address 127.10.242.254:37949)
0618 22:27:26.643685 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:26.643699 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:26.643741 (+ 42us) server_negotiation.cc:407] Connection header received
0618 22:27:26.646060 (+ 2319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:26.646086 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:26.646601 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:26.646979 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:26.647541 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:26.648856 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:26.650224 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:26.650977 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:26.651142 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:26.652269 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:26.652276 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:26.652295 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:26.652925 (+ 630us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:26.653021 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:26.653053 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:26.653189 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:26.653375 (+ 186us) server_negotiation.cc:299] Negotiation successful
0618 22:27:26.653700 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":346,"threads_started":1}
I20260618 22:27:26.658576 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:27:26.663522 16627 raft_consensus.cc:348] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b756a371f1d24b0da3afb630ba2b19b4" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 37949 } }
I20260618 22:27:26.663879 16627 raft_consensus.cc:374] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:26.663949 16627 raft_consensus.cc:729] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b756a371f1d24b0da3afb630ba2b19b4, State: Initialized, Role: FOLLOWER
I20260618 22:27:26.664480 16627 consensus_queue.cc:260] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b756a371f1d24b0da3afb630ba2b19b4" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 37949 } }
I20260618 22:27:26.664666 16627 raft_consensus.cc:388] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:27:26.664758 16627 raft_consensus.cc:482] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:27:26.664872 16627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:26.666927 16627 raft_consensus.cc:504] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b756a371f1d24b0da3afb630ba2b19b4" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 37949 } }
I20260618 22:27:26.667373 16627 leader_election.cc:302] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b756a371f1d24b0da3afb630ba2b19b4; no voters:
I20260618 22:27:26.667887 16627 leader_election.cc:288] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:27:26.668056 16632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:26.669031 16632 raft_consensus.cc:686] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4 [term 1 LEADER]: Becoming Leader. State: Replica: b756a371f1d24b0da3afb630ba2b19b4, State: Running, Role: LEADER
I20260618 22:27:26.669598 16632 consensus_queue.cc:237] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b756a371f1d24b0da3afb630ba2b19b4" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 37949 } }
I20260618 22:27:26.669961 16627 sys_catalog.cc:564] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:27:26.672286 16633 sys_catalog.cc:455] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b756a371f1d24b0da3afb630ba2b19b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b756a371f1d24b0da3afb630ba2b19b4" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 37949 } } }
I20260618 22:27:26.672613 16633 sys_catalog.cc:458] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:26.673414 16640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:27:26.673229 16634 sys_catalog.cc:455] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b756a371f1d24b0da3afb630ba2b19b4. Latest consensus state: current_term: 1 leader_uuid: "b756a371f1d24b0da3afb630ba2b19b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b756a371f1d24b0da3afb630ba2b19b4" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 37949 } } }
I20260618 22:27:26.673563 16634 sys_catalog.cc:458] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:26.678870 16640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:27:26.687674 16640 catalog_manager.cc:1347] Generated new cluster ID: 3d26611ed7d5406295daeba4372ab3bf
I20260618 22:27:26.687801 16640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:27:26.706398 16640 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:27:26.707657 16640 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:27:26.715235 16640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b756a371f1d24b0da3afb630ba2b19b4: Generated new TSK 0
I20260618 22:27:26.716101 16640 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:27:26.744813 16630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:26.736560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42672 (local address 127.10.242.254:37949)
0618 22:27:26.736751 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:26.736758 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:26.737042 (+ 284us) server_negotiation.cc:407] Connection header received
0618 22:27:26.737305 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:26.737311 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:26.737401 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:26.737565 (+ 164us) server_negotiation.cc:226] Negotiated authn=JWT
0618 22:27:26.737976 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:26.739112 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:26.740464 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:26.741165 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:26.741285 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:26.741855 (+ 570us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0618 22:27:26.744168 (+ 2313us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37703/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0618 22:27:26.744505 (+ 337us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42672: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37703/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":76}
W20260618 22:27:26.744957 16630 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42672: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37703/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260618 22:27:26.746508 16656 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:26.736381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:37949 (local address 127.0.0.1:42672)
0618 22:27:26.736945 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:26.736962 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:26.737113 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:26.737567 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:26.737571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:26.737580 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0618 22:27:26.737816 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:26.737825 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:26.739306 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:26.739310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:26.740336 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:26.740348 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:26.741408 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:26.741414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:26.741572 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:26.741607 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0618 22:27:26.744380 (+ 2773us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37703/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0618 22:27:26.746115 (+ 1735us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.242.254:37949: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37703/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":1468,"thread_start_us":291,"threads_started":1}
I20260618 22:27:26.747548 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 16561
I20260618 22:27:26.748672 16622 generic_service.cc:196] Checking for leaks (request via RPC)
E20260618 22:27:26.756089 16657 webserver.cc:533] Webserver: sslize failed:
E20260618 22:27:26.756198 16657 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260618 22:27:26.841238 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 16561
2026-06-18T22:27:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (699 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-18T22:27:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:27:26Z Disabled control of system clock
I20260618 22:27:26.916201 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:37439
--webserver_interface=127.10.242.254
--webserver_port=0
--builtin_ntp_servers=127.10.242.212:35297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:37439
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/testchainca.cert with env {}
W20260618 22:27:27.228201 16662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:27.228562 16662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:27.228655 16662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:27.235877 16662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:27:27.235996 16662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:27.236033 16662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:27.236088 16662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:27:27.236129 16662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260618 22:27:27.236187 16662 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260618 22:27:27.236225 16662 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260618 22:27:27.246186 16662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:35297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:37439
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:37439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:27.247442 16662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:27.248654 16662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:27.260089 16667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:27.260057 16670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:27.260125 16668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:27.261670 16662 server_base.cc:1034] running on GCE node
I20260618 22:27:27.262485 16662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:27.263924 16662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:27.265172 16662 hybrid_clock.cc:648] HybridClock initialized: now 1781821647265123 us; error 44 us; skew 500 ppm
I20260618 22:27:27.265489 16662 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:27:27.267750 16662 webserver.cc:460] Webserver started at http://127.10.242.254:37879/ using document root <none> and password file <none>
I20260618 22:27:27.268406 16662 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:27.268494 16662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:27.268743 16662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:27.271678 16662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "923cf42275da4e4b88d8747b64a17050"
format_stamp: "Formatted at 2026-06-18 22:27:27 on dist-test-slave-0956"
server_key: "346424e527cf61e72b264bec8f23e753"
server_key_iv: "5af1fb31d1db8c9c356aa812739758f9"
server_key_version: "encryptionkey@0"
I20260618 22:27:27.272480 16662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "923cf42275da4e4b88d8747b64a17050"
format_stamp: "Formatted at 2026-06-18 22:27:27 on dist-test-slave-0956"
server_key: "346424e527cf61e72b264bec8f23e753"
server_key_iv: "5af1fb31d1db8c9c356aa812739758f9"
server_key_version: "encryptionkey@0"
I20260618 22:27:27.277490 16662 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260618 22:27:27.280786 16676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:27.282204 16662 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:27:27.282373 16662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "923cf42275da4e4b88d8747b64a17050"
format_stamp: "Formatted at 2026-06-18 22:27:27 on dist-test-slave-0956"
server_key: "346424e527cf61e72b264bec8f23e753"
server_key_iv: "5af1fb31d1db8c9c356aa812739758f9"
server_key_version: "encryptionkey@0"
I20260618 22:27:27.282562 16662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:27.297492 16662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:27.298271 16662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:27.298508 16662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:27.315099 16662 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:37439
I20260618 22:27:27.315117 16727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:37439 every 8 connection(s)
I20260618 22:27:27.316478 16662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:27:27.320511 16728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:27.323339 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 16662
I20260618 22:27:27.323536 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:27:27.324280 11211 external_mini_cluster.cc:1428] Setting key 1e4e0ecf0de54bcd010c61c6a509cd79
I20260618 22:27:27.332294 16728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050: Bootstrap starting.
I20260618 22:27:27.335445 16728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:27.336517 16728 log.cc:824] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:27.337383 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:27.326296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35770 (local address 127.10.242.254:37439)
0618 22:27:27.327238 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:27.327259 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:27.327302 (+ 43us) server_negotiation.cc:407] Connection header received
0618 22:27:27.328964 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:27.328998 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:27.329544 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:27.329908 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:27.330493 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:27.331722 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:27.333004 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:27.333719 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:27.333891 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:27.335036 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:27.335043 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:27.335058 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:27:27.335629 (+ 571us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:27.335728 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:27:27.335752 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:27.335865 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:27.336073 (+ 208us) server_negotiation.cc:299] Negotiation successful
0618 22:27:27.336363 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":318,"threads_started":1}
I20260618 22:27:27.339521 16728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050: No bootstrap required, opened a new log
I20260618 22:27:27.341223 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:27:27.347247 16728 raft_consensus.cc:348] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923cf42275da4e4b88d8747b64a17050" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 37439 } }
I20260618 22:27:27.347597 16728 raft_consensus.cc:374] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:27.347683 16728 raft_consensus.cc:729] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 923cf42275da4e4b88d8747b64a17050, State: Initialized, Role: FOLLOWER
I20260618 22:27:27.348348 16728 consensus_queue.cc:260] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923cf42275da4e4b88d8747b64a17050" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 37439 } }
I20260618 22:27:27.348575 16728 raft_consensus.cc:388] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:27:27.348702 16728 raft_consensus.cc:482] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:27:27.348851 16728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:27.350934 16728 raft_consensus.cc:504] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923cf42275da4e4b88d8747b64a17050" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 37439 } }
I20260618 22:27:27.351419 16728 leader_election.cc:302] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 923cf42275da4e4b88d8747b64a17050; no voters:
I20260618 22:27:27.351953 16728 leader_election.cc:288] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:27:27.352094 16733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:27.352700 16733 raft_consensus.cc:686] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050 [term 1 LEADER]: Becoming Leader. State: Replica: 923cf42275da4e4b88d8747b64a17050, State: Running, Role: LEADER
I20260618 22:27:27.353350 16733 consensus_queue.cc:237] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923cf42275da4e4b88d8747b64a17050" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 37439 } }
I20260618 22:27:27.353994 16728 sys_catalog.cc:564] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:27:27.355880 16735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 923cf42275da4e4b88d8747b64a17050. Latest consensus state: current_term: 1 leader_uuid: "923cf42275da4e4b88d8747b64a17050" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923cf42275da4e4b88d8747b64a17050" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 37439 } } }
I20260618 22:27:27.356220 16735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:27.356055 16734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "923cf42275da4e4b88d8747b64a17050" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923cf42275da4e4b88d8747b64a17050" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 37439 } } }
I20260618 22:27:27.356312 16734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:27.358098 16741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:27:27.362943 16741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:27:27.372349 16741 catalog_manager.cc:1347] Generated new cluster ID: f1cfa4050b7b4954873063181ecd77ca
I20260618 22:27:27.372460 16741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:27:27.385320 16741 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:27:27.387149 16741 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:27:27.399092 16741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 923cf42275da4e4b88d8747b64a17050: Generated new TSK 0
I20260618 22:27:27.400210 16741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:27:27.848389 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:27.413902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35774 (local address 127.10.242.254:37439)
0618 22:27:27.414111 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:27.414122 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:27.414400 (+ 278us) server_negotiation.cc:407] Connection header received
0618 22:27:27.414620 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:27.414624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:27.414698 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:27.414840 (+ 142us) server_negotiation.cc:226] Negotiated authn=JWT
0618 22:27:27.415360 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:27.416078 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:27.418109 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:27.418669 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:27.418776 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:27.419379 (+ 603us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0618 22:27:27.847794 (+428415us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0618 22:27:27.848157 (+ 363us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35774: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":88}
W20260618 22:27:27.848522 16731 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35774: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260618 22:27:27.848827 16757 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:27.413664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:37439 (local address 127.0.0.1:35774)
0618 22:27:27.414274 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:27.414293 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:27.414447 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:27.414874 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:27.414879 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:27.414893 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0618 22:27:27.415195 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:27.415207 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:27.416209 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:27.416216 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:27.417944 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:27.417961 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:27.418836 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:27.418843 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:27.419027 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:27.419073 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0618 22:27:27.848131 (+429058us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0618 22:27:27.848443 (+ 312us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.242.254:37439: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":321,"threads_started":1}
I20260618 22:27:27.849928 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 16662
I20260618 22:27:27.851331 16723 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:27.951479 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 16662
2026-06-18T22:27:27Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1090 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (398 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (392 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:27:28 dist-test-slave-0956 krb5kdc[16772](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:27:28 dist-test-slave-0956 krb5kdc[16772](info): set up 2 sockets
Jun 18 22:27:28 dist-test-slave-0956 krb5kdc[16772](info): commencing operation
krb5kdc: starting...
W20260618 22:27:29.741519 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.004s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:27:29 dist-test-slave-0956 krb5kdc[16772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:27:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:27:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:27:29.944401 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:34703
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:34703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:30.256387 16785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:30.256678 16785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:30.256786 16785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:30.263895 16785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:27:30.264000 16785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:30.264048 16785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:30.264123 16785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:27:30.264175 16785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:27:30.273818 16785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:34703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:34703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:30.275090 16785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:30.276348 16785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:30.291692 16790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:30.291759 16793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:30.291774 16791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:30.293042 16785 server_base.cc:1034] running on GCE node
I20260618 22:27:30.293804 16785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:30.295156 16785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:30.296375 16785 hybrid_clock.cc:648] HybridClock initialized: now 1781821650296358 us; error 59 us; skew 500 ppm
I20260618 22:27:30.296708 16785 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:30 dist-test-slave-0956 krb5kdc[16772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821650, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:30.299679 16785 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:27:30.301028 16785 webserver.cc:460] Webserver started at http://127.10.242.254:39655/ using document root <none> and password file <none>
I20260618 22:27:30.301738 16785 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:30.301855 16785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:30.302134 16785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:30.304890 16785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "e83b6da3875f40e9bdc80f41da9b44b3"
format_stamp: "Formatted at 2026-06-18 22:27:30 on dist-test-slave-0956"
server_key: "6386311a32dc962d93e1868c8c0afe1c"
server_key_iv: "56f5aedb3e356f3d6055c6f3bd07d659"
server_key_version: "encryptionkey@0"
I20260618 22:27:30.305627 16785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "e83b6da3875f40e9bdc80f41da9b44b3"
format_stamp: "Formatted at 2026-06-18 22:27:30 on dist-test-slave-0956"
server_key: "6386311a32dc962d93e1868c8c0afe1c"
server_key_iv: "56f5aedb3e356f3d6055c6f3bd07d659"
server_key_version: "encryptionkey@0"
I20260618 22:27:30.310463 16785 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260618 22:27:30.313934 16800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:30.315201 16785 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:27:30.315353 16785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "e83b6da3875f40e9bdc80f41da9b44b3"
format_stamp: "Formatted at 2026-06-18 22:27:30 on dist-test-slave-0956"
server_key: "6386311a32dc962d93e1868c8c0afe1c"
server_key_iv: "56f5aedb3e356f3d6055c6f3bd07d659"
server_key_version: "encryptionkey@0"
I20260618 22:27:30.315513 16785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:30.328611 16785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:30.332595 16785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:30.332846 16785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:30.348003 16785 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:34703
I20260618 22:27:30.348033 16851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:34703 every 8 connection(s)
I20260618 22:27:30.349314 16785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:27:30.352703 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 16785
I20260618 22:27:30.352914 16852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:30.352932 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:27:30.353554 11211 external_mini_cluster.cc:1428] Setting key 49ac1b3018f6bc07b9cbaca6a620d436
I20260618 22:27:30.365986 16852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3: Bootstrap starting.
Jun 18 22:27:30 dist-test-slave-0956 krb5kdc[16772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:30.369508 16852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:30.370512 16852 log.cc:824] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:30.373159 16852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3: No bootstrap required, opened a new log
I20260618 22:27:30.376336 16855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:30.355399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35178 (local address 127.10.242.254:34703)
0618 22:27:30.356185 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:30.356196 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:30.356235 (+ 39us) server_negotiation.cc:407] Connection header received
0618 22:27:30.358477 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:30.358502 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:30.358913 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:30.359262 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:30.360930 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:30.362248 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:30.364064 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:30.365076 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:30.365294 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:30.369564 (+ 4270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:30.369575 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:30.369594 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:30.369638 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:30.372449 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:30.373107 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:30.373117 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:30.373125 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:30.373330 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:30.373695 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:30.373700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:30.373704 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:30.374212 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:30.374484 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:30.374945 (+ 461us) server_negotiation.cc:299] Negotiation successful
0618 22:27:30.375353 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":366,"threads_started":1}
I20260618 22:27:30.380903 16852 raft_consensus.cc:348] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83b6da3875f40e9bdc80f41da9b44b3" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34703 } }
I20260618 22:27:30.381307 16852 raft_consensus.cc:374] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:30.381403 16852 raft_consensus.cc:729] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e83b6da3875f40e9bdc80f41da9b44b3, State: Initialized, Role: FOLLOWER
I20260618 22:27:30.381933 16852 consensus_queue.cc:260] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83b6da3875f40e9bdc80f41da9b44b3" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34703 } }
I20260618 22:27:30.382139 16852 raft_consensus.cc:388] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:27:30.382262 16852 raft_consensus.cc:482] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:27:30.382397 16852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:30.384356 16852 raft_consensus.cc:504] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83b6da3875f40e9bdc80f41da9b44b3" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34703 } }
I20260618 22:27:30.384799 16852 leader_election.cc:302] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e83b6da3875f40e9bdc80f41da9b44b3; no voters:
I20260618 22:27:30.385429 16852 leader_election.cc:288] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:27:30.385529 16857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:30.386142 16857 raft_consensus.cc:686] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3 [term 1 LEADER]: Becoming Leader. State: Replica: e83b6da3875f40e9bdc80f41da9b44b3, State: Running, Role: LEADER
I20260618 22:27:30.386637 16857 consensus_queue.cc:237] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83b6da3875f40e9bdc80f41da9b44b3" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34703 } }
I20260618 22:27:30.387305 16852 sys_catalog.cc:564] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:27:30.389225 16858 sys_catalog.cc:455] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e83b6da3875f40e9bdc80f41da9b44b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83b6da3875f40e9bdc80f41da9b44b3" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34703 } } }
I20260618 22:27:30.389346 16859 sys_catalog.cc:455] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e83b6da3875f40e9bdc80f41da9b44b3. Latest consensus state: current_term: 1 leader_uuid: "e83b6da3875f40e9bdc80f41da9b44b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83b6da3875f40e9bdc80f41da9b44b3" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34703 } } }
I20260618 22:27:30.389593 16858 sys_catalog.cc:458] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:30.389618 16859 sys_catalog.cc:458] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:30.391268 16865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:27:30.396224 16865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:27:30.405071 16865 catalog_manager.cc:1347] Generated new cluster ID: a71ff46e16b241b2b08f6e641dbd7419
I20260618 22:27:30.405215 16865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:27:30.447146 16865 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:27:30.448968 16865 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:27:30.471918 16865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e83b6da3875f40e9bdc80f41da9b44b3: Generated new TSK 0
I20260618 22:27:30.472995 16865 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
I20260618 22:27:30.522512 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:34703
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:30.835606 16878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:30.835909 16878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:30.836026 16878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:30.843278 16878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:30.843406 16878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:30.843564 16878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:27:30.855293 16878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:34703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:30.856599 16878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:30.857910 16878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:30.870913 16883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:30.870968 16886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:30.870951 16884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:30.871699 16878 server_base.cc:1034] running on GCE node
I20260618 22:27:30.872320 16878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:30.873279 16878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:30.874492 16878 hybrid_clock.cc:648] HybridClock initialized: now 1781821650874431 us; error 54 us; skew 500 ppm
I20260618 22:27:30.874789 16878 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:30 dist-test-slave-0956 krb5kdc[16772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821650, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:30.877966 16878 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:27:30.879559 16878 webserver.cc:460] Webserver started at http://127.10.242.193:33733/ using document root <none> and password file <none>
I20260618 22:27:30.880293 16878 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:30.880399 16878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:30.880702 16878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:30.883737 16878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "31fc1a1a74314301be109ca0a6857ead"
format_stamp: "Formatted at 2026-06-18 22:27:30 on dist-test-slave-0956"
server_key: "63826826f4410bef3641a01312b3db98"
server_key_iv: "50a573d54f53767d052a0218f7f51f3b"
server_key_version: "encryptionkey@0"
I20260618 22:27:30.884503 16878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "31fc1a1a74314301be109ca0a6857ead"
format_stamp: "Formatted at 2026-06-18 22:27:30 on dist-test-slave-0956"
server_key: "63826826f4410bef3641a01312b3db98"
server_key_iv: "50a573d54f53767d052a0218f7f51f3b"
server_key_version: "encryptionkey@0"
I20260618 22:27:30.889712 16878 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260618 22:27:30.893414 16893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:30.894897 16878 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:27:30.895066 16878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "31fc1a1a74314301be109ca0a6857ead"
format_stamp: "Formatted at 2026-06-18 22:27:30 on dist-test-slave-0956"
server_key: "63826826f4410bef3641a01312b3db98"
server_key_iv: "50a573d54f53767d052a0218f7f51f3b"
server_key_version: "encryptionkey@0"
I20260618 22:27:30.895262 16878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:30.922995 16878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:30.927273 16878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:30.927549 16878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:30.928527 16878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:30.929970 16878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:30.930047 16878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:30.930142 16878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:30.930202 16878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:30.960781 16878 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:45797
I20260618 22:27:30.960855 17005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:45797 every 8 connection(s)
I20260618 22:27:30.962028 16878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:27:30.970844 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 16878
I20260618 22:27:30.971062 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:27:30.971820 11211 external_mini_cluster.cc:1428] Setting key 49a8420cde6b21c51c6b8a393899f1b2
Jun 18 22:27:30 dist-test-slave-0956 krb5kdc[16772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821650, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:30.982091 17008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:30.964539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:46605 (local address 127.10.242.254:34703)
0618 22:27:30.964932 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:30.964937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:30.965810 (+ 873us) server_negotiation.cc:407] Connection header received
0618 22:27:30.967588 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:30.967594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:30.967680 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:30.967771 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:30.969344 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:30.970048 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:30.972257 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:30.972674 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:30.972777 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:30.976525 (+ 3748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:30.976531 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:30.976535 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:30.976571 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:30.979150 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:30.979884 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:30.979888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:30.979890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:30.980018 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:30.980420 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:30.980424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:30.980426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:30.980692 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:30.980831 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:30.981541 (+ 710us) server_negotiation.cc:299] Negotiation successful
0618 22:27:30.981751 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":199,"threads_started":1}
I20260618 22:27:30.983141 17009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:30.964943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34703 (local address 127.10.242.193:46605)
0618 22:27:30.965649 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:30.965689 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:30.967325 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:30.968021 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:30.968028 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:30.968534 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:30.969166 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:30.969186 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:30.970192 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:30.970196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:30.972119 (+ 1923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:30.972133 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:30.972891 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:30.972897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:30.973128 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:30.973758 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:30.973781 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:30.976259 (+ 2478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:30.979286 (+ 3027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:30.979294 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:30.979314 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:30.979755 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:30.980120 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:30.980123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:30.980126 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:30.980304 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:30.980821 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:30.980828 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:30.981191 (+ 363us) client_negotiation.cc:769] Sending connection context
0618 22:27:30.981429 (+ 238us) client_negotiation.cc:240] Negotiation successful
0618 22:27:30.981974 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":295,"threads_started":1}
I20260618 22:27:30.984735 17006 heartbeater.cc:347] Connected to a master server at 127.10.242.254:34703
I20260618 22:27:30.985019 17006 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:30.985735 17006 heartbeater.cc:511] Master 127.10.242.254:34703 requested a full tablet report, sending...
I20260618 22:27:30.988019 16817 ts_manager.cc:194] Registered new tserver with Master: 31fc1a1a74314301be109ca0a6857ead (127.10.242.193:45797)
I20260618 22:27:30.990094 16817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:46605
WARNING: no policy specified for kudu/127.10.242.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
I20260618 22:27:31.013628 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:34703
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:31.325368 17012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:31.325629 17012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:31.325718 17012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:31.332831 17012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:31.332950 17012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:31.333062 17012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:27:31.342720 17012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:34703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:31.343950 17012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:31.345220 17012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:31.357656 17017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:31.357749 17020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:31.357982 17018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:31.358719 17012 server_base.cc:1034] running on GCE node
I20260618 22:27:31.359287 17012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:31.360477 17012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:31.361730 17012 hybrid_clock.cc:648] HybridClock initialized: now 1781821651361675 us; error 51 us; skew 500 ppm
I20260618 22:27:31.362026 17012 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:31 dist-test-slave-0956 krb5kdc[16772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821651, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:31.364979 17012 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:27:31.366401 17012 webserver.cc:460] Webserver started at http://127.10.242.194:44147/ using document root <none> and password file <none>
I20260618 22:27:31.367034 17012 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:31.367132 17012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:31.367422 17012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:31.370337 17012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "7230a691e3934cbe8caf9458a40165c3"
format_stamp: "Formatted at 2026-06-18 22:27:31 on dist-test-slave-0956"
server_key: "37634ae8e516906d6274ad420dfd0ba3"
server_key_iv: "63920d8762fea920f2bebc7264862c29"
server_key_version: "encryptionkey@0"
I20260618 22:27:31.371121 17012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "7230a691e3934cbe8caf9458a40165c3"
format_stamp: "Formatted at 2026-06-18 22:27:31 on dist-test-slave-0956"
server_key: "37634ae8e516906d6274ad420dfd0ba3"
server_key_iv: "63920d8762fea920f2bebc7264862c29"
server_key_version: "encryptionkey@0"
I20260618 22:27:31.376289 17012 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:27:31.379913 17027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:31.381300 17012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:27:31.381438 17012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "7230a691e3934cbe8caf9458a40165c3"
format_stamp: "Formatted at 2026-06-18 22:27:31 on dist-test-slave-0956"
server_key: "37634ae8e516906d6274ad420dfd0ba3"
server_key_iv: "63920d8762fea920f2bebc7264862c29"
server_key_version: "encryptionkey@0"
I20260618 22:27:31.381618 17012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:31.395347 17012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:31.399525 17012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:31.399863 17012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:31.400873 17012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:31.402199 17012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:31.402273 17012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:31.402391 17012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:31.402451 17012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:31.432258 17012 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:46269
I20260618 22:27:31.432307 17139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:46269 every 8 connection(s)
I20260618 22:27:31.433535 17012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:27:31.441851 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 17012
I20260618 22:27:31.442036 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:27:31.442407 11211 external_mini_cluster.cc:1428] Setting key 1d4960c2cf3cba47485e876827d72189
Jun 18 22:27:31 dist-test-slave-0956 krb5kdc[16772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821651, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:31.452620 17008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:31.436006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:57599 (local address 127.10.242.254:34703)
0618 22:27:31.436172 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:31.436177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:31.437332 (+ 1155us) server_negotiation.cc:407] Connection header received
0618 22:27:31.439078 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:31.439083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:31.439159 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:31.439243 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:31.440711 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:31.441467 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:31.443079 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:31.443589 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:31.443709 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:31.447095 (+ 3386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:31.447103 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:31.447108 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:31.447162 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:31.449803 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:31.450473 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:31.450477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:31.450480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:31.450600 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:31.451006 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:31.451010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:31.451012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:31.451259 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:31.451394 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:31.452087 (+ 693us) server_negotiation.cc:299] Negotiation successful
0618 22:27:31.452303 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:27:31.453653 17142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:31.436416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34703 (local address 127.10.242.194:57599)
0618 22:27:31.437179 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:31.437218 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:31.438807 (+ 1589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:31.439475 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:31.439487 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:31.439992 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:31.440544 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:31.440562 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:31.441619 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:31.441623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:31.442935 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:31.442948 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:31.443783 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:31.443788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:31.444043 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:31.444675 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:31.444698 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:31.446801 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:31.449941 (+ 3140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:31.449951 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:31.449969 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:31.450361 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:31.450699 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:31.450704 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:31.450708 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:31.450896 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:31.451376 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:31.451384 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:31.451737 (+ 353us) client_negotiation.cc:769] Sending connection context
0618 22:27:31.451976 (+ 239us) client_negotiation.cc:240] Negotiation successful
0618 22:27:31.452509 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":264,"threads_started":1}
I20260618 22:27:31.455052 17140 heartbeater.cc:347] Connected to a master server at 127.10.242.254:34703
I20260618 22:27:31.455386 17140 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:31.456241 17140 heartbeater.cc:511] Master 127.10.242.254:34703 requested a full tablet report, sending...
I20260618 22:27:31.457839 16817 ts_manager.cc:194] Registered new tserver with Master: 7230a691e3934cbe8caf9458a40165c3 (127.10.242.194:46269)
I20260618 22:27:31.458640 16817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:57599
WARNING: no policy specified for kudu/127.10.242.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
I20260618 22:27:31.484565 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:34703
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:31.792510 17145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:31.792778 17145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:31.792898 17145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:31.800320 17145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:31.800439 17145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:31.800607 17145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:27:31.810441 17145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:34703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:31.811722 17145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:31.813031 17145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:31.827677 17150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:31.827726 17153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:31.827754 17151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:31.828508 17145 server_base.cc:1034] running on GCE node
I20260618 22:27:31.829152 17145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:31.830091 17145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:31.831297 17145 hybrid_clock.cc:648] HybridClock initialized: now 1781821651831224 us; error 67 us; skew 500 ppm
I20260618 22:27:31.831629 17145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:31 dist-test-slave-0956 krb5kdc[16772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821651, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:31.834756 17145 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:27:31.836252 17145 webserver.cc:460] Webserver started at http://127.10.242.195:40969/ using document root <none> and password file <none>
I20260618 22:27:31.836925 17145 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:31.837046 17145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:31.837373 17145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:31.840236 17145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "742c90d92f344fd1b48e97ef1e7ba135"
format_stamp: "Formatted at 2026-06-18 22:27:31 on dist-test-slave-0956"
server_key: "57f0597c643fcf57bcd5080acf23fddd"
server_key_iv: "367c04273ba6da4cd05bf97224cee249"
server_key_version: "encryptionkey@0"
I20260618 22:27:31.840974 17145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "742c90d92f344fd1b48e97ef1e7ba135"
format_stamp: "Formatted at 2026-06-18 22:27:31 on dist-test-slave-0956"
server_key: "57f0597c643fcf57bcd5080acf23fddd"
server_key_iv: "367c04273ba6da4cd05bf97224cee249"
server_key_version: "encryptionkey@0"
I20260618 22:27:31.846241 17145 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260618 22:27:31.849977 17160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:31.851512 17145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:27:31.851672 17145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "742c90d92f344fd1b48e97ef1e7ba135"
format_stamp: "Formatted at 2026-06-18 22:27:31 on dist-test-slave-0956"
server_key: "57f0597c643fcf57bcd5080acf23fddd"
server_key_iv: "367c04273ba6da4cd05bf97224cee249"
server_key_version: "encryptionkey@0"
I20260618 22:27:31.851872 17145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:31.878409 17145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:31.882454 17145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:31.882714 17145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:31.883634 17145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:31.884979 17145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:31.885064 17145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:31.885208 17145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:31.885262 17145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:31.914341 17145 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:35029
I20260618 22:27:31.914407 17272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:35029 every 8 connection(s)
I20260618 22:27:31.915563 17145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:27:31.922859 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 17145
I20260618 22:27:31.923058 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:27:31.923491 11211 external_mini_cluster.cc:1428] Setting key 7dda73564e15e57d96ff2220e509d7f7
Jun 18 22:27:31 dist-test-slave-0956 krb5kdc[16772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821651, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:31.935756 17008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:31.918085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:48027 (local address 127.10.242.254:34703)
0618 22:27:31.918281 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:31.918286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:31.919471 (+ 1185us) server_negotiation.cc:407] Connection header received
0618 22:27:31.921360 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:31.921365 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:31.921447 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:31.921538 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:31.923262 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:31.924063 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:31.926112 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:31.926688 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:31.926791 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:31.930178 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:31.930187 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:31.930191 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:31.930226 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:31.932967 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:31.933751 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:31.933756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:31.933758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:31.933881 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:31.934237 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:31.934240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:31.934242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:31.934500 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:31.934626 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:31.935314 (+ 688us) server_negotiation.cc:299] Negotiation successful
0618 22:27:31.935485 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260618 22:27:31.936774 17275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:31.918550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34703 (local address 127.10.242.195:48027)
0618 22:27:31.919308 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:31.919363 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:31.921061 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:31.921787 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:31.921795 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:31.922308 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:31.923074 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:31.923101 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:31.924257 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:31.924261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:31.925953 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:31.925967 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:31.926903 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:31.926909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:31.927108 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:31.927746 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:31.927769 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:31.929952 (+ 2183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:31.933165 (+ 3213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:31.933172 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:31.933186 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:31.933617 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:31.933962 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:31.933965 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:31.933967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:31.934155 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:31.934644 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:31.934650 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:31.934996 (+ 346us) client_negotiation.cc:769] Sending connection context
0618 22:27:31.935228 (+ 232us) client_negotiation.cc:240] Negotiation successful
0618 22:27:31.935631 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":281,"threads_started":1}
I20260618 22:27:31.938107 17273 heartbeater.cc:347] Connected to a master server at 127.10.242.254:34703
I20260618 22:27:31.938385 17273 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:31.939121 17273 heartbeater.cc:511] Master 127.10.242.254:34703 requested a full tablet report, sending...
I20260618 22:27:31.940455 16816 ts_manager.cc:194] Registered new tserver with Master: 742c90d92f344fd1b48e97ef1e7ba135 (127.10.242.195:35029)
I20260618 22:27:31.941313 16816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:48027
I20260618 22:27:31.951020 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/test-admin.keytab.
Jun 18 22:27:31 dist-test-slave-0956 krb5kdc[16772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:31.977998 11211 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260618 22:27:31.982898 17280 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:31.980563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33802 (local address 127.10.242.193:45797)
0618 22:27:31.981179 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:31.981186 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:31.981209 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:27:31.981265 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:31.981272 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:31.981481 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:31.981620 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:31.982511 (+ 891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33802: BlockingRecv error: recv got EOF from 127.0.0.1:33802 (error 108)
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":287,"threads_started":1}
W20260618 22:27:31.983088 11211 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/krb5cc' of type FILE
Jun 18 22:27:31 dist-test-slave-0956 krb5kdc[16772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:31.984582 11211 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 18 22:27:31 dist-test-slave-0956 krb5kdc[16772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:31.986001 11211 init.cc:295] Successfully reacquired a new kerberos TGT
I20260618 22:27:31.993162 17006 heartbeater.cc:503] Master 127.10.242.254:34703 was elected leader, sending a full tablet report...
Jun 18 22:27:31 dist-test-slave-0956 krb5kdc[16772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:27:32.000732 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:31.987626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33804 (local address 127.10.242.193:45797)
0618 22:27:31.987789 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:31.987795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:31.987951 (+ 156us) server_negotiation.cc:407] Connection header received
0618 22:27:31.988125 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:31.988129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:31.988205 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:31.988275 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:31.989251 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:31.990181 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:31.991291 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:31.991830 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:31.991917 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:31.995469 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:31.995480 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:31.995496 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:31.995537 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:31.998147 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:31.998812 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:31.998819 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:31.998821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:31.998939 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:31.999281 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:31.999286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:31.999289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:31.999633 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:31.999898 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:32.000209 (+ 311us) server_negotiation.cc:299] Negotiation successful
0618 22:27:32.000452 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260618 22:27:32.006049 17280 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:32.004347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33818 (local address 127.10.242.193:45797)
0618 22:27:32.004500 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:32.004533 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:32.004821 (+ 288us) server_negotiation.cc:407] Connection header received
0618 22:27:32.004948 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:32.004952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:32.005007 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:32.005116 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:32.005827 (+ 711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33818: BlockingRecv error: recv got EOF from 127.0.0.1:33818 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260618 22:27:32.006273 11211 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/krb5cc' of type FILE
Jun 18 22:27:32 dist-test-slave-0956 krb5kdc[16772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:32.007762 11211 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 18 22:27:32 dist-test-slave-0956 krb5kdc[16772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:32.009030 11211 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 18 22:27:32 dist-test-slave-0956 krb5kdc[16772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:27:32.022006 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:32.010300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33830 (local address 127.10.242.193:45797)
0618 22:27:32.010437 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:32.010440 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:32.010753 (+ 313us) server_negotiation.cc:407] Connection header received
0618 22:27:32.010884 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:32.010887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:32.010940 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:32.011017 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:32.011857 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:32.012537 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:32.013762 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:32.014178 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:32.014265 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:32.017479 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:32.017485 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:32.017488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:32.017520 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:32.019812 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:32.020416 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:32.020420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:32.020422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:32.020555 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:32.020866 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:32.020869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:32.020871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:32.021168 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:32.021354 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:32.021590 (+ 236us) server_negotiation.cc:299] Negotiation successful
0618 22:27:32.021739 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260618 22:27:32.025800 17280 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:32.024328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33836 (local address 127.10.242.193:45797)
0618 22:27:32.024509 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:32.024514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:32.024743 (+ 229us) server_negotiation.cc:407] Connection header received
0618 22:27:32.024852 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:32.024856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:32.024919 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:32.024990 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:32.025610 (+ 620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33836: BlockingRecv error: recv got EOF from 127.0.0.1:33836 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260618 22:27:32.026257 11211 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821604371818-11211-0/krb5kdc/krb5cc)
Jun 18 22:27:32 dist-test-slave-0956 krb5kdc[16772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:32.027603 11211 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 18 22:27:32 dist-test-slave-0956 krb5kdc[16772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:32.028745 11211 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 18 22:27:32 dist-test-slave-0956 krb5kdc[16772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:27:32.043076 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:32.030008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33852 (local address 127.10.242.193:45797)
0618 22:27:32.030184 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:32.030188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:32.030378 (+ 190us) server_negotiation.cc:407] Connection header received
0618 22:27:32.030466 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:32.030470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:32.030524 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:32.030606 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:32.031587 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:32.032472 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:32.033703 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:32.034123 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:32.034216 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:32.037649 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:32.037657 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:32.037662 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:32.037705 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:32.040938 (+ 3233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:32.041559 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:32.041563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:32.041565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:32.041708 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:32.041981 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:32.041984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:32.041986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:32.042223 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:32.042434 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:32.042579 (+ 145us) server_negotiation.cc:299] Negotiation successful
0618 22:27:32.042730 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:27:32.044384 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 16878
I20260618 22:27:32.055099 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:32.045027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33858 (local address 127.10.242.193:45797)
0618 22:27:32.045185 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:32.045188 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:32.045521 (+ 333us) server_negotiation.cc:407] Connection header received
0618 22:27:32.045629 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:32.045632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:32.045682 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:32.045751 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:32.046578 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:32.047214 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:32.048378 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:32.048809 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:32.048895 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:32.050707 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:32.050712 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:32.050715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:32.050742 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:32.052992 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:32.053582 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:32.053586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:32.053588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:32.053700 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:32.053972 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:32.053975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:32.053976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:32.054201 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:32.054372 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:32.054657 (+ 285us) server_negotiation.cc:299] Negotiation successful
0618 22:27:32.054813 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:27:32.055622 17001 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:32.136814 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 16878
I20260618 22:27:32.149955 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 17012
Jun 18 22:27:32 dist-test-slave-0956 krb5kdc[16772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.194@KRBTEST.COM
I20260618 22:27:32.165001 17293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:32.150812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57974 (local address 127.10.242.194:46269)
0618 22:27:32.151281 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:32.151288 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:32.151311 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:27:32.151415 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:32.151422 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:32.151629 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:32.151770 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:32.152721 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:32.153742 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:32.155023 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:32.155934 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:32.156043 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:32.159731 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:32.159742 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:32.159757 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:32.159792 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:32.162537 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:32.163202 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:32.163209 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:32.163211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:32.163323 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:32.163664 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:32.163667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:32.163669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:32.163989 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:32.164234 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:32.164469 (+ 235us) server_negotiation.cc:299] Negotiation successful
0618 22:27:32.164681 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":236,"threads_started":1}
I20260618 22:27:32.166239 17135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:32.247731 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 17012
I20260618 22:27:32.259811 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 17145
Jun 18 22:27:32 dist-test-slave-0956 krb5kdc[16772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.195@KRBTEST.COM
I20260618 22:27:32.274679 17295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:32.260801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35900 (local address 127.10.242.195:35029)
0618 22:27:32.261495 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:32.261502 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:32.261536 (+ 34us) server_negotiation.cc:407] Connection header received
0618 22:27:32.261594 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:32.261600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:32.261800 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:32.261921 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:32.262848 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:32.263842 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:32.265036 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:32.265603 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:32.265695 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:32.269453 (+ 3758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:32.269465 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:32.269480 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:32.269514 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:32.272157 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:32.272867 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:32.272874 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:32.272876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:32.272982 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:32.273317 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:32.273320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:32.273322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:32.273649 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:32.273888 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:32.274149 (+ 261us) server_negotiation.cc:299] Negotiation successful
0618 22:27:32.274364 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":404,"threads_started":1}
I20260618 22:27:32.275979 17268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:32.361725 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 17145
I20260618 22:27:32.373792 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 16785
I20260618 22:27:32.375068 16847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:32.475008 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 16785
2026-06-18T22:27:32Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3736 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:27:32 dist-test-slave-0956 krb5kdc[17300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:27:32 dist-test-slave-0956 krb5kdc[17300](info): set up 2 sockets
Jun 18 22:27:32 dist-test-slave-0956 krb5kdc[17300](info): commencing operation
krb5kdc: starting...
W20260618 22:27:33.473625 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.003s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:27:33 dist-test-slave-0956 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:27:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:27:33Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.254.keytab.
Entry for principal oryx/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.254.keytab.
I20260618 22:27:33.673460 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:34635
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.254.keytab
--principal=oryx/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:34635
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:33.985628 17313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:33.985916 17313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:33.986037 17313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:33.993427 17313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:27:33.993541 17313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:33.993598 17313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:33.993672 17313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:27:33.993729 17313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:27:34.003870 17313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:34635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.254.keytab
--principal=oryx/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:34635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:34.005232 17313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:34.006505 17313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:34.017776 17319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:34.017755 17318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:34.018208 17321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:34.018507 17313 server_base.cc:1034] running on GCE node
I20260618 22:27:34.019223 17313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:34.020592 17313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:34.021787 17313 hybrid_clock.cc:648] HybridClock initialized: now 1781821654021733 us; error 45 us; skew 500 ppm
I20260618 22:27:34.022106 17313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:34 dist-test-slave-0956 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821654, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:34.025141 17313 init.cc:377] Logged in from keytab as oryx/127.10.242.254@KRBTEST.COM (short username oryx)
I20260618 22:27:34.026537 17313 webserver.cc:460] Webserver started at http://127.10.242.254:46775/ using document root <none> and password file <none>
I20260618 22:27:34.027202 17313 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:34.027315 17313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:34.027590 17313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:34.030496 17313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "7a255bfe590d478991390a7b3b7e9b80"
format_stamp: "Formatted at 2026-06-18 22:27:34 on dist-test-slave-0956"
server_key: "df6d37269267f64972757a638cf7dc98"
server_key_iv: "da2fae49e6cd17991f2568ec03ae15f5"
server_key_version: "encryptionkey@0"
I20260618 22:27:34.031210 17313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "7a255bfe590d478991390a7b3b7e9b80"
format_stamp: "Formatted at 2026-06-18 22:27:34 on dist-test-slave-0956"
server_key: "df6d37269267f64972757a638cf7dc98"
server_key_iv: "da2fae49e6cd17991f2568ec03ae15f5"
server_key_version: "encryptionkey@0"
I20260618 22:27:34.036438 17313 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:27:34.039954 17328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:34.041374 17313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:27:34.041536 17313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "7a255bfe590d478991390a7b3b7e9b80"
format_stamp: "Formatted at 2026-06-18 22:27:34 on dist-test-slave-0956"
server_key: "df6d37269267f64972757a638cf7dc98"
server_key_iv: "da2fae49e6cd17991f2568ec03ae15f5"
server_key_version: "encryptionkey@0"
I20260618 22:27:34.041713 17313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:34.063035 17313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:34.067179 17313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:34.067519 17313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:34.084247 17313 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:34635
I20260618 22:27:34.084275 17379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:34635 every 8 connection(s)
I20260618 22:27:34.085780 17313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:27:34.089377 17380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:34.092880 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 17313
I20260618 22:27:34.093124 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:27:34.093921 11211 external_mini_cluster.cc:1428] Setting key f5471d0cb84ddc63585f5049a6ddf6b2
I20260618 22:27:34.102463 17380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80: Bootstrap starting.
I20260618 22:27:34.106271 17380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:34.107576 17380 log.cc:824] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:27:34 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM
I20260618 22:27:34.110509 17380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80: No bootstrap required, opened a new log
I20260618 22:27:34.116106 17383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:34.095969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44722 (local address 127.10.242.254:34635)
0618 22:27:34.096740 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:34.096752 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:34.096792 (+ 40us) server_negotiation.cc:407] Connection header received
0618 22:27:34.098343 (+ 1551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:34.098367 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:34.098806 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:34.099152 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:34.100831 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:34.102234 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:34.103933 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:34.104637 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:34.104769 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:34.109150 (+ 4381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:34.109161 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:34.109178 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:34.109218 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:34.112113 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:34.112779 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:34.112785 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:34.112795 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:34.112959 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:34.113375 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:34.113378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:34.113381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:34.113925 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:34.114203 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:34.114676 (+ 473us) server_negotiation.cc:299] Negotiation successful
0618 22:27:34.115085 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":343,"threads_started":1}
I20260618 22:27:34.118510 17380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } }
I20260618 22:27:34.118914 17380 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:34.119057 17380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a255bfe590d478991390a7b3b7e9b80, State: Initialized, Role: FOLLOWER
I20260618 22:27:34.119608 17380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } }
I20260618 22:27:34.119822 17380 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:27:34.119939 17380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:27:34.120086 17380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:34.122349 17380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } }
I20260618 22:27:34.122813 17380 leader_election.cc:302] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a255bfe590d478991390a7b3b7e9b80; no voters:
I20260618 22:27:34.123317 17380 leader_election.cc:288] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:27:34.123436 17385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:34.124030 17385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [term 1 LEADER]: Becoming Leader. State: Replica: 7a255bfe590d478991390a7b3b7e9b80, State: Running, Role: LEADER
I20260618 22:27:34.124531 17385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } }
I20260618 22:27:34.125370 17380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:27:34.127287 17386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a255bfe590d478991390a7b3b7e9b80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } }
I20260618 22:27:34.127633 17386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:34.127923 17387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a255bfe590d478991390a7b3b7e9b80. Latest consensus state: current_term: 1 leader_uuid: "7a255bfe590d478991390a7b3b7e9b80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } }
I20260618 22:27:34.128331 17387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:34.129936 17394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:27:34.134398 17394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:27:34.142447 17394 catalog_manager.cc:1347] Generated new cluster ID: c4e1d5b328594abba209eac27d127622
I20260618 22:27:34.142575 17394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:27:34.162355 17394 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:27:34.164331 17394 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:27:34.176200 17394 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80: Generated new TSK 0
I20260618 22:27:34.177313 17394 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.193.keytab.
Entry for principal oryx/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.193.keytab.
I20260618 22:27:34.224970 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:34635
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.193.keytab
--principal=oryx/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:34.545676 17406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:34.545996 17406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:34.546183 17406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:34.553531 17406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:34.553655 17406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:34.553807 17406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:27:34.563513 17406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.193.keytab
--principal=oryx/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:34635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:34.564791 17406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:34.566195 17406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:34.578991 17412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:34.579028 17414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:34.579053 17411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:34.579538 17406 server_base.cc:1034] running on GCE node
I20260618 22:27:34.580173 17406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:34.581063 17406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:34.582315 17406 hybrid_clock.cc:648] HybridClock initialized: now 1781821654582266 us; error 41 us; skew 500 ppm
I20260618 22:27:34.582630 17406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:34 dist-test-slave-0956 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821654, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:34.585667 17406 init.cc:377] Logged in from keytab as oryx/127.10.242.193@KRBTEST.COM (short username oryx)
I20260618 22:27:34.587056 17406 webserver.cc:460] Webserver started at http://127.10.242.193:34705/ using document root <none> and password file <none>
I20260618 22:27:34.587700 17406 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:34.587810 17406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:34.588088 17406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:34.591044 17406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "5899712e99ff4d88bb415b836db9967c"
format_stamp: "Formatted at 2026-06-18 22:27:34 on dist-test-slave-0956"
server_key: "99dd079ae1f3ef21e8aa2510debf6c0c"
server_key_iv: "d9d755156c41140d06854032b0fce6e3"
server_key_version: "encryptionkey@0"
I20260618 22:27:34.591777 17406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "5899712e99ff4d88bb415b836db9967c"
format_stamp: "Formatted at 2026-06-18 22:27:34 on dist-test-slave-0956"
server_key: "99dd079ae1f3ef21e8aa2510debf6c0c"
server_key_iv: "d9d755156c41140d06854032b0fce6e3"
server_key_version: "encryptionkey@0"
I20260618 22:27:34.596961 17406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260618 22:27:34.600332 17421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:34.601686 17406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:27:34.601833 17406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "5899712e99ff4d88bb415b836db9967c"
format_stamp: "Formatted at 2026-06-18 22:27:34 on dist-test-slave-0956"
server_key: "99dd079ae1f3ef21e8aa2510debf6c0c"
server_key_iv: "d9d755156c41140d06854032b0fce6e3"
server_key_version: "encryptionkey@0"
I20260618 22:27:34.602015 17406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:34.621236 17406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:34.625066 17406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:34.625368 17406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:34.626313 17406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:34.627624 17406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:34.627693 17406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:34.627781 17406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:34.627821 17406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:34.657292 17406 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:46667
I20260618 22:27:34.657315 17533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:46667 every 8 connection(s)
I20260618 22:27:34.658545 17406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:27:34.666522 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 17406
I20260618 22:27:34.666741 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:27:34.667327 11211 external_mini_cluster.cc:1428] Setting key b3f72db0cbd9c50bc2800f3af4954626
Jun 18 22:27:34 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821654, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM
I20260618 22:27:34.677968 17536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:34.660864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:33387 (local address 127.10.242.254:34635)
0618 22:27:34.661279 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:34.661283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:34.662279 (+ 996us) server_negotiation.cc:407] Connection header received
0618 22:27:34.664013 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:34.664018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:34.664096 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:34.664189 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:34.665680 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:34.666543 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:34.668253 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:34.668805 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:34.668935 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:34.672383 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:34.672388 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:34.672391 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:34.672424 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:34.674765 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:34.675600 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:34.675604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:34.675606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:34.675721 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:34.676135 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:34.676139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:34.676141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:34.676517 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:34.676656 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:34.677381 (+ 725us) server_negotiation.cc:299] Negotiation successful
0618 22:27:34.677618 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":198,"threads_started":1}
I20260618 22:27:34.678925 17537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:34.661457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.193:33387)
0618 22:27:34.662126 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:34.662171 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:34.663763 (+ 1592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:34.664426 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:34.664434 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:34.664921 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:34.665524 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:34.665541 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:34.666727 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:34.666731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:34.668111 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:34.668124 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:34.669042 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:34.669050 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:34.669351 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:34.670004 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:34.670030 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:34.672145 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:34.674912 (+ 2767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:34.674922 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:34.674943 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:34.675467 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:34.675823 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:34.675828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:34.675832 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:34.676018 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:34.676643 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:34.676651 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:34.676991 (+ 340us) client_negotiation.cc:769] Sending connection context
0618 22:27:34.677281 (+ 290us) client_negotiation.cc:240] Negotiation successful
0618 22:27:34.677821 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":246,"threads_started":1}
I20260618 22:27:34.680337 17534 heartbeater.cc:347] Connected to a master server at 127.10.242.254:34635
I20260618 22:27:34.680676 17534 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:34.681545 17534 heartbeater.cc:511] Master 127.10.242.254:34635 requested a full tablet report, sending...
I20260618 22:27:34.683724 17345 ts_manager.cc:194] Registered new tserver with Master: 5899712e99ff4d88bb415b836db9967c (127.10.242.193:46667)
WARNING: no policy specified for oryx/127.10.242.194@KRBTEST.COM; defaulting to no policy
I20260618 22:27:34.685192 17345 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:33387
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.194.keytab.
Entry for principal oryx/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.194.keytab.
I20260618 22:27:34.709602 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:34635
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.194.keytab
--principal=oryx/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:35.023605 17540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:35.023908 17540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:35.024041 17540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:35.031908 17540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:35.032040 17540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:35.032207 17540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:27:35.042074 17540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.194.keytab
--principal=oryx/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:34635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:35.043298 17540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:35.044565 17540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:35.062525 17545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:35.062602 17546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:35.062554 17548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:35.063786 17540 server_base.cc:1034] running on GCE node
I20260618 22:27:35.064352 17540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:35.065280 17540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:35.066480 17540 hybrid_clock.cc:648] HybridClock initialized: now 1781821655066430 us; error 39 us; skew 500 ppm
I20260618 22:27:35.066807 17540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:35 dist-test-slave-0956 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821655, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:35.074343 17540 init.cc:377] Logged in from keytab as oryx/127.10.242.194@KRBTEST.COM (short username oryx)
I20260618 22:27:35.075814 17540 webserver.cc:460] Webserver started at http://127.10.242.194:40421/ using document root <none> and password file <none>
I20260618 22:27:35.076468 17540 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:35.076586 17540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:35.076871 17540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:35.079833 17540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "0fb54f92e5ac4441bed6f5bb6ea32144"
format_stamp: "Formatted at 2026-06-18 22:27:35 on dist-test-slave-0956"
server_key: "8a8997b1e0f78f0bfa443f3788b37f91"
server_key_iv: "2b5f4f3e0ed5fb4aeec698ece7e69ed0"
server_key_version: "encryptionkey@0"
I20260618 22:27:35.080578 17540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "0fb54f92e5ac4441bed6f5bb6ea32144"
format_stamp: "Formatted at 2026-06-18 22:27:35 on dist-test-slave-0956"
server_key: "8a8997b1e0f78f0bfa443f3788b37f91"
server_key_iv: "2b5f4f3e0ed5fb4aeec698ece7e69ed0"
server_key_version: "encryptionkey@0"
I20260618 22:27:35.085843 17540 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260618 22:27:35.089411 17555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:35.090878 17540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:27:35.091022 17540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "0fb54f92e5ac4441bed6f5bb6ea32144"
format_stamp: "Formatted at 2026-06-18 22:27:35 on dist-test-slave-0956"
server_key: "8a8997b1e0f78f0bfa443f3788b37f91"
server_key_iv: "2b5f4f3e0ed5fb4aeec698ece7e69ed0"
server_key_version: "encryptionkey@0"
I20260618 22:27:35.091199 17540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:35.112799 17540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:35.116834 17540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:35.117130 17540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:35.118076 17540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:35.119433 17540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:35.119510 17540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:35.119632 17540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:35.119683 17540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:35.150034 17540 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:42531
I20260618 22:27:35.150089 17667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:42531 every 8 connection(s)
I20260618 22:27:35.151245 17540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:27:35.159696 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 17540
I20260618 22:27:35.159883 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:27:35.160318 11211 external_mini_cluster.cc:1428] Setting key a0a3bd9bcadda521d06e151da29955bb
Jun 18 22:27:35 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821655, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM
I20260618 22:27:35.170778 17536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:35.153610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:59315 (local address 127.10.242.254:34635)
0618 22:27:35.153762 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:35.153766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:35.154878 (+ 1112us) server_negotiation.cc:407] Connection header received
0618 22:27:35.156627 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:35.156633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:35.156724 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:35.156810 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:35.158308 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:35.159022 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:35.160659 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:35.161207 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:35.161322 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:35.164773 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:35.164778 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:35.164783 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:35.164820 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:35.167768 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:35.168515 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:35.168521 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:35.168524 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:35.168682 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:35.169058 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:35.169062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:35.169066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:35.169438 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:35.169585 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:35.170247 (+ 662us) server_negotiation.cc:299] Negotiation successful
0618 22:27:35.170468 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:27:35.171713 17670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:35.154056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.194:59315)
0618 22:27:35.154719 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:35.154762 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:35.156362 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:35.157059 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:35.157068 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:35.157592 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:35.158146 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:35.158163 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:35.159160 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:35.159164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:35.160523 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:35.160535 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:35.161421 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:35.161428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:35.161645 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:35.162306 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:35.162329 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:35.164540 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:35.167945 (+ 3405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:35.167953 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:35.167967 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:35.168404 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:35.168787 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:35.168790 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:35.168792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:35.168967 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:35.169553 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:35.169559 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:35.169904 (+ 345us) client_negotiation.cc:769] Sending connection context
0618 22:27:35.170136 (+ 232us) client_negotiation.cc:240] Negotiation successful
0618 22:27:35.170531 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":247,"threads_started":1}
I20260618 22:27:35.173100 17668 heartbeater.cc:347] Connected to a master server at 127.10.242.254:34635
I20260618 22:27:35.173415 17668 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:35.174268 17668 heartbeater.cc:511] Master 127.10.242.254:34635 requested a full tablet report, sending...
I20260618 22:27:35.175894 17345 ts_manager.cc:194] Registered new tserver with Master: 0fb54f92e5ac4441bed6f5bb6ea32144 (127.10.242.194:42531)
I20260618 22:27:35.176779 17345 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:59315
WARNING: no policy specified for oryx/127.10.242.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.195.keytab.
Entry for principal oryx/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.195.keytab.
I20260618 22:27:35.205811 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:34635
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.195.keytab
--principal=oryx/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:35.514981 17673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:35.515282 17673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:35.515427 17673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:35.522879 17673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:35.522991 17673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:35.523146 17673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:27:35.533169 17673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.195.keytab
--principal=oryx/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:34635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:35.534466 17673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:35.535825 17673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:35.549333 17678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:35.549273 17681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:35.549314 17679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:35.549881 17673 server_base.cc:1034] running on GCE node
I20260618 22:27:35.550508 17673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:35.551401 17673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:35.552627 17673 hybrid_clock.cc:648] HybridClock initialized: now 1781821655552557 us; error 59 us; skew 500 ppm
I20260618 22:27:35.552968 17673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:35 dist-test-slave-0956 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821655, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:35.556151 17673 init.cc:377] Logged in from keytab as oryx/127.10.242.195@KRBTEST.COM (short username oryx)
I20260618 22:27:35.557614 17673 webserver.cc:460] Webserver started at http://127.10.242.195:34349/ using document root <none> and password file <none>
I20260618 22:27:35.558305 17673 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:35.558436 17673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:35.558730 17673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:35.561965 17673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "b4a2f32ef55149f2b75701234c139912"
format_stamp: "Formatted at 2026-06-18 22:27:35 on dist-test-slave-0956"
server_key: "9f7e8151ef21f614951dfe05f2a905c6"
server_key_iv: "90089658278b2ca66a3f511adef10b57"
server_key_version: "encryptionkey@0"
I20260618 22:27:35.562759 17673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "b4a2f32ef55149f2b75701234c139912"
format_stamp: "Formatted at 2026-06-18 22:27:35 on dist-test-slave-0956"
server_key: "9f7e8151ef21f614951dfe05f2a905c6"
server_key_iv: "90089658278b2ca66a3f511adef10b57"
server_key_version: "encryptionkey@0"
I20260618 22:27:35.568336 17673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:27:35.572212 17688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:35.573725 17673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:27:35.573899 17673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "b4a2f32ef55149f2b75701234c139912"
format_stamp: "Formatted at 2026-06-18 22:27:35 on dist-test-slave-0956"
server_key: "9f7e8151ef21f614951dfe05f2a905c6"
server_key_iv: "90089658278b2ca66a3f511adef10b57"
server_key_version: "encryptionkey@0"
I20260618 22:27:35.574093 17673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:35.604223 17673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:35.608739 17673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:35.609053 17673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:35.610086 17673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:35.611580 17673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:35.611668 17673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:35.611799 17673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:35.611864 17673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:35.642936 17673 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:46165
I20260618 22:27:35.642969 17800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:46165 every 8 connection(s)
I20260618 22:27:35.644372 17673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:27:35.654718 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 17673
I20260618 22:27:35.654877 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:27:35.655284 11211 external_mini_cluster.cc:1428] Setting key b554ab7bc50bdc3ebf37d42fd8832fec
Jun 18 22:27:35 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821655, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM
I20260618 22:27:35.664451 17536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:35.646943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:54677 (local address 127.10.242.254:34635)
0618 22:27:35.647138 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:35.647142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:35.648261 (+ 1119us) server_negotiation.cc:407] Connection header received
0618 22:27:35.650063 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:35.650069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:35.650155 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:35.650236 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:35.651675 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:35.652414 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:35.653943 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:35.654367 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:35.654491 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:35.658339 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:35.658345 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:35.658350 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:35.658390 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:35.661646 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:35.662365 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:35.662370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:35.662374 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:35.662530 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:35.662922 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:35.662926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:35.662929 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:35.663252 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:35.663371 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:35.663988 (+ 617us) server_negotiation.cc:299] Negotiation successful
0618 22:27:35.664157 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260618 22:27:35.665336 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:35.647438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.195:54677)
0618 22:27:35.648110 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:35.648151 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:35.649803 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:35.650470 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:35.650478 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:35.650965 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:35.651514 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:35.651531 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:35.652547 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:35.652551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:35.653818 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:35.653830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:35.654599 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:35.654605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:35.654813 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:35.655475 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:35.655496 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:35.658108 (+ 2612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:35.661802 (+ 3694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:35.661808 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:35.661820 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:35.662231 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:35.662629 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:35.662632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:35.662635 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:35.662806 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:35.663349 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:35.663355 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:35.663649 (+ 294us) client_negotiation.cc:769] Sending connection context
0618 22:27:35.663851 (+ 202us) client_negotiation.cc:240] Negotiation successful
0618 22:27:35.664228 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":262,"threads_started":1}
I20260618 22:27:35.666565 17801 heartbeater.cc:347] Connected to a master server at 127.10.242.254:34635
I20260618 22:27:35.666834 17801 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:35.667537 17801 heartbeater.cc:511] Master 127.10.242.254:34635 requested a full tablet report, sending...
I20260618 22:27:35.669030 17345 ts_manager.cc:194] Registered new tserver with Master: b4a2f32ef55149f2b75701234c139912 (127.10.242.195:46165)
I20260618 22:27:35.670073 17345 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:54677
I20260618 22:27:35.670634 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:27:35.688194 17534 heartbeater.cc:503] Master 127.10.242.254:34635 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.10.242.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.253.keytab.
Entry for principal oryx/127.10.242.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.253.keytab.
I20260618 22:27:35.712440 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.253:33505
--webserver_interface=127.10.242.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.253.keytab
--principal=oryx/127.10.242.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:34635,127.10.242.253:33505
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:36.025007 17806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:36.025362 17806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:36.025489 17806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:36.032797 17806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:27:36.032914 17806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:36.032963 17806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:36.033038 17806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:27:36.033119 17806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:27:36.042876 17806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:34635,127.10.242.253:33505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.253.keytab
--principal=oryx/127.10.242.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.253:33505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.242.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
W20260618 22:27:36.043253 17806 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.10.242.254:34635,127.10.242.253:33505'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260618 22:27:36.044301 17806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:36.045634 17806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:36.057022 17812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:36.057310 17811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:36.057508 17814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:36.058104 17806 server_base.cc:1034] running on GCE node
I20260618 22:27:36.059068 17806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:36.060636 17806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:36.061848 17806 hybrid_clock.cc:648] HybridClock initialized: now 1781821656061782 us; error 49 us; skew 500 ppm
I20260618 22:27:36.062178 17806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:36 dist-test-slave-0956 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821656, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:36.065128 17806 init.cc:377] Logged in from keytab as oryx/127.10.242.253@KRBTEST.COM (short username oryx)
I20260618 22:27:36.066686 17806 webserver.cc:460] Webserver started at http://127.10.242.253:34093/ using document root <none> and password file <none>
I20260618 22:27:36.067364 17806 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:36.067452 17806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:36.067723 17806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:36.070637 17806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/data/instance:
uuid: "468d5c19d3c54b7bbd0d7122ea722111"
format_stamp: "Formatted at 2026-06-18 22:27:36 on dist-test-slave-0956"
server_key: "a20f4f747ec6964e89a23e51e7627c72"
server_key_iv: "768bb8c48c5fcc0695b4e236629a3ba8"
server_key_version: "encryptionkey@0"
I20260618 22:27:36.071386 17806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/wal/instance:
uuid: "468d5c19d3c54b7bbd0d7122ea722111"
format_stamp: "Formatted at 2026-06-18 22:27:36 on dist-test-slave-0956"
server_key: "a20f4f747ec6964e89a23e51e7627c72"
server_key_iv: "768bb8c48c5fcc0695b4e236629a3ba8"
server_key_version: "encryptionkey@0"
I20260618 22:27:36.076354 17806 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260618 22:27:36.079828 17821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:36.081192 17806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:27:36.081326 17806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/wal
uuid: "468d5c19d3c54b7bbd0d7122ea722111"
format_stamp: "Formatted at 2026-06-18 22:27:36 on dist-test-slave-0956"
server_key: "a20f4f747ec6964e89a23e51e7627c72"
server_key_iv: "768bb8c48c5fcc0695b4e236629a3ba8"
server_key_version: "encryptionkey@0"
I20260618 22:27:36.081498 17806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:36.132485 17806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:36.136668 17806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:36.136893 17806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:36.154762 17806 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.253:33505
I20260618 22:27:36.154779 17872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.253:33505 every 8 connection(s)
I20260618 22:27:36.156030 17806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/data/info.pb
I20260618 22:27:36.159633 17873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:36.161119 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 17806
I20260618 22:27:36.168318 17873 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:27:36 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.253@KRBTEST.COM
Jun 18 22:27:36 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821656, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.253@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM
I20260618 22:27:36.179471 17668 heartbeater.cc:503] Master 127.10.242.254:34635 was elected leader, sending a full tablet report...
I20260618 22:27:36.183739 17876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.163295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60038 (local address 127.10.242.253:33505)
0618 22:27:36.164068 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:36.164083 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:36.164133 (+ 50us) server_negotiation.cc:407] Connection header received
0618 22:27:36.165707 (+ 1574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:36.165729 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:36.166124 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:36.166485 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:36.167894 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.169265 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.170589 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.171517 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.171707 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.175749 (+ 4042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:36.175760 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:36.175776 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:36.175824 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:36.179274 (+ 3450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.180051 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.180059 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.180068 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.180230 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.180624 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.180629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.180632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.181332 (+ 700us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:36.181610 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:36.182013 (+ 403us) server_negotiation.cc:299] Negotiation successful
0618 22:27:36.182454 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":358,"threads_started":1}
I20260618 22:27:36.185508 17878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.170836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.0.0.1:33964)
0618 22:27:36.171619 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:36.171663 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:36.171799 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:36.172184 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:36.172191 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:36.172746 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:36.172961 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.172969 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.173963 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.173967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.175163 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.175177 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.176007 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.176014 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.176150 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.176739 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:36.176761 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:36.179330 (+ 2569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:36.183011 (+ 3681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.183019 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.183024 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.183424 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.183788 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.183791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.183793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.183938 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.184511 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:36.184517 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:36.184688 (+ 171us) client_negotiation.cc:769] Sending connection context
0618 22:27:36.184909 (+ 221us) client_negotiation.cc:240] Negotiation successful
0618 22:27:36.185203 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":401,"threads_started":1}
I20260618 22:27:36.185562 17877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.170802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33964 (local address 127.10.242.254:34635)
0618 22:27:36.171234 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:36.171240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:36.171897 (+ 657us) server_negotiation.cc:407] Connection header received
0618 22:27:36.171958 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:36.171962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:36.172020 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:36.172115 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:36.173134 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.173834 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.175326 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.175833 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.175970 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.179544 (+ 3574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:36.179551 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:36.179556 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:36.179608 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:36.182884 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.183530 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.183536 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.183539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.183688 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.184060 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.184064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.184066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.184398 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:36.184503 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:36.185006 (+ 503us) server_negotiation.cc:299] Negotiation successful
0618 22:27:36.185241 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":205,"threads_started":1}
I20260618 22:27:36.187685 17873 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:27:36 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821656, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.253@KRBTEST.COM for oryx/127.10.242.253@KRBTEST.COM
I20260618 22:27:36.198956 17878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.188373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:33505 (local address 127.0.0.1:60042)
0618 22:27:36.188583 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:36.188598 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:36.188685 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:36.188953 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:36.188958 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:36.189322 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:36.189439 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.189447 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.190359 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.190364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.191194 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.191203 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.191791 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.191796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.191952 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.192405 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:36.192421 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:36.194468 (+ 2047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:36.196837 (+ 2369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.196842 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.196846 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.197232 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.197520 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.197525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.197529 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.197647 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.198088 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:36.198093 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:36.198221 (+ 128us) client_negotiation.cc:769] Sending connection context
0618 22:27:36.198320 (+ 99us) client_negotiation.cc:240] Negotiation successful
0618 22:27:36.198569 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260618 22:27:36.199023 17876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.188444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60042 (local address 127.10.242.253:33505)
0618 22:27:36.188583 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:36.188587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:36.188651 (+ 64us) server_negotiation.cc:407] Connection header received
0618 22:27:36.188757 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:36.188761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:36.188815 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:36.188900 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:36.189552 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.190236 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.191310 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.191667 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.191768 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.194630 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:36.194635 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:36.194640 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:36.194672 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:36.196728 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.197326 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.197331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.197335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.197433 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.197740 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.197745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.197749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.197991 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:36.198111 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:36.198417 (+ 306us) server_negotiation.cc:299] Negotiation successful
0618 22:27:36.198698 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":53}
I20260618 22:27:36.203449 17873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: Bootstrap starting.
I20260618 22:27:36.206415 17873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:36.207347 17873 log.cc:824] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:36.209890 17873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: No bootstrap required, opened a new log
I20260618 22:27:36.216377 17873 raft_consensus.cc:348] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } }
I20260618 22:27:36.216697 17873 raft_consensus.cc:374] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:36.216779 17873 raft_consensus.cc:729] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468d5c19d3c54b7bbd0d7122ea722111, State: Initialized, Role: FOLLOWER
I20260618 22:27:36.217320 17873 consensus_queue.cc:260] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } }
I20260618 22:27:36.218024 17880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } } }
I20260618 22:27:36.218286 17880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:27:36.218921 17873 sys_catalog.cc:564] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [sys.catalog]: configured and running, proceeding with master startup.
W20260618 22:27:36.226635 17891 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:27:36.226824 17891 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:27:36.229507 17806 master_runner.cc:428] Detected that this master 468d5c19d3c54b7bbd0d7122ea722111 is joining an existing cluster
I20260618 22:27:36.229596 17806 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.242.253:33505
I20260618 22:27:36.249687 17897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.233499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.0.0.1:33970)
0618 22:27:36.234421 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:36.234439 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:36.234615 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:36.234940 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:36.234945 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:36.235450 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:36.235737 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.235746 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.236752 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.236758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.238808 (+ 2050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.238824 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.240327 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.240339 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.240557 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.241332 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:36.241357 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:36.242502 (+ 1145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:36.246344 (+ 3842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.246347 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.246351 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.246759 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.247274 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.247279 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.247283 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.247498 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.248236 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:36.248241 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:36.248492 (+ 251us) client_negotiation.cc:769] Sending connection context
0618 22:27:36.248664 (+ 172us) client_negotiation.cc:240] Negotiation successful
0618 22:27:36.249074 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"mutex_wait_us":4,"thread_start_us":672,"threads_started":1}
I20260618 22:27:36.250720 17877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.233412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33970 (local address 127.10.242.254:34635)
0618 22:27:36.233600 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:36.233606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:36.234519 (+ 913us) server_negotiation.cc:407] Connection header received
0618 22:27:36.234736 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:36.234742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:36.234815 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:36.234894 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:36.235867 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.236620 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.239171 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.239832 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.239964 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.242758 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:36.242766 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:36.242770 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:36.242814 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:36.246174 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.246883 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.246889 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.246893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.247129 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.247666 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.247671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.247675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.248080 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:36.249982 (+ 1902us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:36.250122 (+ 140us) server_negotiation.cc:299] Negotiation successful
0618 22:27:36.250358 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:27:36.261329 17876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.233798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60044 (local address 127.10.242.253:33505)
0618 22:27:36.233978 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:36.233983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:36.245347 (+ 11364us) server_negotiation.cc:407] Connection header received
0618 22:27:36.245490 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:36.245495 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:36.245565 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:36.245649 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:36.247042 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.248151 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.250244 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.250936 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.251034 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.254082 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:36.254090 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:36.254094 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:36.254130 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:36.257588 (+ 3458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.258491 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.258497 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.258501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.258683 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.259522 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.259528 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.259532 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.259928 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:36.260108 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:36.260651 (+ 543us) server_negotiation.cc:299] Negotiation successful
0618 22:27:36.260881 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
WARNING: no policy specified for oryx/127.10.242.252@KRBTEST.COM; defaulting to no policy
I20260618 22:27:36.262431 17898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.233567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:33505 (local address 127.0.0.1:60044)
0618 22:27:36.245244 (+ 11677us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:36.245260 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:36.245385 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:36.245802 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:36.245806 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:36.246506 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:36.246842 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.246855 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.248356 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.248361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.250094 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.250108 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.251321 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.252025 (+ 704us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.252207 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.252794 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:36.252815 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:36.253908 (+ 1093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:36.257753 (+ 3845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.257759 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.257764 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.258339 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.258831 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.258836 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.258839 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.259377 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.260185 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:36.260189 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:36.260347 (+ 158us) client_negotiation.cc:769] Sending connection context
0618 22:27:36.261614 (+ 1267us) client_negotiation.cc:240] Negotiation successful
0618 22:27:36.261907 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11502,"mutex_wait_us":28,"thread_start_us":11289,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.252@KRBTEST.COM" created.
Jun 18 22:27:36 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821654, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.254@KRBTEST.COM for oryx/127.10.242.253@KRBTEST.COM
I20260618 22:27:36.283849 17876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.257922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60060 (local address 127.10.242.253:33505)
0618 22:27:36.261579 (+ 3657us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:36.261585 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:36.270136 (+ 8551us) server_negotiation.cc:407] Connection header received
0618 22:27:36.270484 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:36.270488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:36.270575 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:36.270677 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:36.271865 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.272582 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.273924 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.274420 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.274551 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.277884 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:36.277897 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:36.277900 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:36.277932 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:36.280803 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.281660 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.281665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.281667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.281801 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.282184 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.282188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.282190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.282465 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:36.282606 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:36.283189 (+ 583us) server_negotiation.cc:299] Negotiation successful
0618 22:27:36.283417 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3561,"thread_start_us":11349,"threads_started":1}
I20260618 22:27:36.283880 17900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.257569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:33505 (local address 127.0.0.1:60060)
0618 22:27:36.269978 (+ 12409us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:36.270031 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:36.270266 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:36.270762 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:36.270770 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:36.271449 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:36.271687 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.271698 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.272715 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.272720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.273781 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.273794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.274594 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.274601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.274788 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.275444 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:36.275464 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:36.277690 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:36.280956 (+ 3266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.280972 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.280981 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.281527 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.281912 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.281918 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.281922 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.282069 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.282597 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:36.282606 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:36.282854 (+ 248us) client_negotiation.cc:769] Sending connection context
0618 22:27:36.283056 (+ 202us) client_negotiation.cc:240] Negotiation successful
0618 22:27:36.283417 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12113,"thread_start_us":11967,"threads_started":1}
I20260618 22:27:36.286042 17345 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.242.253:33505: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111"
member_type: NON_VOTER
last_known_addr {
host: "127.10.242.253"
port: 33505
}
attrs {
promote: true
}
}
dest_uuid: "7a255bfe590d478991390a7b3b7e9b80"
cas_config_opid_index: -1
I20260618 22:27:36.287079 17345 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: NON_VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: true } }
I20260618 22:27:36.290210 17905 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 468d5c19d3c54b7bbd0d7122ea722111 (127.10.242.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: NON_VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: true } } }
I20260618 22:27:36.291656 17848 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:36.293606 17905 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.242.253:33505
I20260618 22:27:36.294212 17904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7a255bfe590d478991390a7b3b7e9b80" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: NON_VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: true } } }
I20260618 22:27:36.294661 17806 master.cc:537] Master@127.10.242.253:33505 shutting down...
I20260618 22:27:36.294612 17904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:36.296037 17848 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 1 FOLLOWER]: Refusing update from remote peer 7a255bfe590d478991390a7b3b7e9b80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260618 22:27:36.297106 17904 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: NON_VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.252.keytab.
Entry for principal oryx/127.10.242.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.252.keytab.
W20260618 22:27:36.300421 17331 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260618 22:27:36.300926 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.252:35943
--webserver_interface=127.10.242.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.252.keytab
--principal=oryx/127.10.242.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:34635,127.10.242.253:33505,127.10.242.252:35943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:27:36.307834 17806 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:36.308367 17806 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:36.308629 17806 tablet_replica.cc:330] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: stopping tablet replica
W20260618 22:27:36.309859 17331 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 -> Peer 468d5c19d3c54b7bbd0d7122ea722111 (127.10.242.253:33505): Couldn't send request to peer 468d5c19d3c54b7bbd0d7122ea722111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260618 22:27:36.315250 17806 master.cc:552] Master@127.10.242.253:33505 shutdown complete.
I20260618 22:27:36.315338 17806 master_runner.cc:305] Clearing existing system tablet
I20260618 22:27:36.315776 17806 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:36.315863 17806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:36.318760 17806 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:27:36.320432 17910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:36.320767 17806 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:36.321023 17806 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260618 22:27:36.321206 17806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/wal
uuid: "468d5c19d3c54b7bbd0d7122ea722111"
format_stamp: "Formatted at 2026-06-18 22:27:36 on dist-test-slave-0956"
server_key: "a20f4f747ec6964e89a23e51e7627c72"
server_key_iv: "768bb8c48c5fcc0695b4e236629a3ba8"
server_key_version: "encryptionkey@0"
I20260618 22:27:36.326146 17806 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:36.332978 17806 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260618 22:27:36.333168 17806 log.cc:1196] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260618 22:27:36.333582 17806 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: Deleting consensus metadata
I20260618 22:27:36.333832 17806 master_runner.cc:315] Copying system tablet from 127.10.242.254:34635
I20260618 22:27:36.341493 17806 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: tablet copy: Beginning tablet copy session from remote peer at address 127.10.242.254:34635
I20260618 22:27:36.359694 17877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.342307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33974 (local address 127.10.242.254:34635)
0618 22:27:36.342495 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:36.342501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:36.345659 (+ 3158us) server_negotiation.cc:407] Connection header received
0618 22:27:36.345852 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:36.345858 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:36.345961 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:36.346088 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:36.347148 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.348283 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.350119 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.350766 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.350901 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.353146 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:36.353153 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:36.353158 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:36.353196 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:36.356560 (+ 3364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.357436 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.357442 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.357446 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.357609 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.358062 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.358067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.358071 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.358450 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:36.358603 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:36.359122 (+ 519us) server_negotiation.cc:299] Negotiation successful
0618 22:27:36.359340 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:27:36.359771 17916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.342246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.0.0.1:33974)
0618 22:27:36.345525 (+ 3279us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:36.345545 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:36.345717 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:36.346205 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:36.346211 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:36.346721 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:36.346986 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.346999 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.348500 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.348507 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.349961 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.349980 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.350954 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.350961 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.351129 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.351834 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:36.351856 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:36.352895 (+ 1039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:36.356709 (+ 3814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.356716 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.356721 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.357311 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.357734 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.357739 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.357743 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.357941 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.358661 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:36.358666 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:36.358836 (+ 170us) client_negotiation.cc:769] Sending connection context
0618 22:27:36.358974 (+ 138us) client_negotiation.cc:240] Negotiation successful
0618 22:27:36.359321 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3067,"thread_start_us":2973,"threads_started":1}
I20260618 22:27:36.361027 17365 tablet_copy_service.cc:140] P 7a255bfe590d478991390a7b3b7e9b80: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 468d5c19d3c54b7bbd0d7122ea722111 ({username='oryx', principal='oryx/127.10.242.253@KRBTEST.COM'} at 127.0.0.1:33974)
I20260618 22:27:36.361349 17365 tablet_copy_service.cc:161] P 7a255bfe590d478991390a7b3b7e9b80: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 468d5c19d3c54b7bbd0d7122ea722111 at {username='oryx', principal='oryx/127.10.242.253@KRBTEST.COM'} at 127.0.0.1:33974: session id = 468d5c19d3c54b7bbd0d7122ea722111-00000000000000000000000000000000
I20260618 22:27:36.365568 17365 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80: Tablet Copy: opened 0 blocks and 1 log segments
I20260618 22:27:36.367771 17806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260618 22:27:36.374402 17806 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: tablet copy: Starting download of 0 data blocks...
I20260618 22:27:36.374696 17806 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: tablet copy: Starting download of 1 WAL segments...
I20260618 22:27:36.377471 17806 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260618 22:27:36.381747 17365 tablet_copy_service.cc:342] P 7a255bfe590d478991390a7b3b7e9b80: Request end of tablet copy session 468d5c19d3c54b7bbd0d7122ea722111-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.242.253@KRBTEST.COM'} at 127.0.0.1:33974
I20260618 22:27:36.381950 17365 tablet_copy_service.cc:433] P 7a255bfe590d478991390a7b3b7e9b80: ending tablet copy session 468d5c19d3c54b7bbd0d7122ea722111-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 468d5c19d3c54b7bbd0d7122ea722111
I20260618 22:27:36.391600 17806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:36.394345 17917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:36.403533 17918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:36.403967 17920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:36.404592 17806 server_base.cc:1034] running on GCE node
I20260618 22:27:36.404871 17806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:36.405475 17806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:36.406666 17806 hybrid_clock.cc:648] HybridClock initialized: now 1781821656406612 us; error 34 us; skew 500 ppm
I20260618 22:27:36.406869 17806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:36 dist-test-slave-0956 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821656, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:36.408497 17806 init.cc:377] Logged in from keytab as oryx/127.10.242.253@KRBTEST.COM (short username oryx)
I20260618 22:27:36.409713 17806 webserver.cc:460] Webserver started at http://127.10.242.253:37867/ using document root <none> and password file <none>
I20260618 22:27:36.410004 17806 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:36.410072 17806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:36.412151 17806 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260618 22:27:36.413681 17927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:36.414045 17806 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260618 22:27:36.414168 17806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/wal
uuid: "468d5c19d3c54b7bbd0d7122ea722111"
format_stamp: "Formatted at 2026-06-18 22:27:36 on dist-test-slave-0956"
server_key: "a20f4f747ec6964e89a23e51e7627c72"
server_key_iv: "768bb8c48c5fcc0695b4e236629a3ba8"
server_key_version: "encryptionkey@0"
I20260618 22:27:36.414309 17806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:36.427379 17806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:36.430663 17806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:36.456117 17806 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.253:33505
I20260618 22:27:36.456137 17978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.253:33505 every 8 connection(s)
I20260618 22:27:36.456931 17806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-1/data/info.pb
I20260618 22:27:36.462327 17979 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:27:36.465355 17979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: Bootstrap starting.
I20260618 22:27:36.477499 17979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:27:36.477903 17979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: Bootstrap complete.
I20260618 22:27:36.478720 17979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: NON_VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: true } }
I20260618 22:27:36.478924 17979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 468d5c19d3c54b7bbd0d7122ea722111, State: Initialized, Role: LEARNER
I20260618 22:27:36.479161 17979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: NON_VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: true } }
I20260618 22:27:36.479883 17982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: NON_VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: true } } }
I20260618 22:27:36.480165 17979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:27:36.480271 17982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [sys.catalog]: This master's current role is: LEARNER
I20260618 22:27:36.490638 17993 catalog_manager.cc:1259] Loaded cluster ID: c4e1d5b328594abba209eac27d127622
I20260618 22:27:36.490724 17993 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: loading cluster ID for follower catalog manager: success
I20260618 22:27:36.492949 17993 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: acquiring CA information for follower catalog manager: success
I20260618 22:27:36.494805 17993 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 22:27:36.672585 17801 heartbeater.cc:503] Master 127.10.242.254:34635 was elected leader, sending a full tablet report...
W20260618 22:27:36.702708 17908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:36.703063 17908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:36.703207 17908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260618 22:27:36.711249 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.704716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60068 (local address 127.10.242.253:33505)
0618 22:27:36.705269 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:36.705273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:36.705289 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:27:36.705345 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:36.705348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:36.705413 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:36.705514 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:36.705982 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.707005 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.709349 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.710158 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.710292 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.710432 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:36.710807 (+ 375us) server_negotiation.cc:299] Negotiation successful
0618 22:27:36.710914 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":289,"threads_started":1}
I20260618 22:27:36.711246 17900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.704461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:33505 (local address 127.0.0.1:60068)
0618 22:27:36.704677 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:36.704700 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:36.704841 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:36.705588 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:36.705594 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:36.705626 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:36.705836 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.705847 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.707141 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.707147 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.709216 (+ 2069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.709234 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.710323 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.710330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.710599 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.710691 (+ 92us) client_negotiation.cc:769] Sending connection context
0618 22:27:36.710792 (+ 101us) client_negotiation.cc:240] Negotiation successful
0618 22:27:36.710958 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
W20260618 22:27:36.712944 17908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:27:36.713064 17908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
I20260618 22:27:36.712443 17954 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
W20260618 22:27:36.713168 17908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:36.713256 17908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:27:36.713328 17908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:27:36.713601 17982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a255bfe590d478991390a7b3b7e9b80. Latest consensus state: current_term: 1 leader_uuid: "7a255bfe590d478991390a7b3b7e9b80" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: NON_VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: true } } }
I20260618 22:27:36.713845 17982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [sys.catalog]: This master's current role is: LEARNER
I20260618 22:27:36.724141 17908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:34635,127.10.242.253:33505,127.10.242.252:35943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.252.keytab
--principal=oryx/127.10.242.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.252:35943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.242.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:36.725555 17908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:36.726886 17908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:36.741703 18000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:36.742340 18002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:36.744818 17999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:36.745895 17908 server_base.cc:1034] running on GCE node
I20260618 22:27:36.746762 17908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:36.748327 17908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:36.749542 17908 hybrid_clock.cc:648] HybridClock initialized: now 1781821656749489 us; error 50 us; skew 500 ppm
I20260618 22:27:36.749905 17908 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:36 dist-test-slave-0956 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821656, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:36.753036 17908 init.cc:377] Logged in from keytab as oryx/127.10.242.252@KRBTEST.COM (short username oryx)
I20260618 22:27:36.754521 17908 webserver.cc:460] Webserver started at http://127.10.242.252:33799/ using document root <none> and password file <none>
I20260618 22:27:36.755256 17908 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:36.755355 17908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:36.755645 17908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:36.759032 17908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/data/instance:
uuid: "bfd01e9042ff4ee3819c16bcee8dd973"
format_stamp: "Formatted at 2026-06-18 22:27:36 on dist-test-slave-0956"
server_key: "2173903d37a9e19be5aa05bbc1645e6f"
server_key_iv: "8fb9d72eb10d3b7e9f8757c6b6b15f0e"
server_key_version: "encryptionkey@0"
I20260618 22:27:36.759790 17908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/wal/instance:
uuid: "bfd01e9042ff4ee3819c16bcee8dd973"
format_stamp: "Formatted at 2026-06-18 22:27:36 on dist-test-slave-0956"
server_key: "2173903d37a9e19be5aa05bbc1645e6f"
server_key_iv: "8fb9d72eb10d3b7e9f8757c6b6b15f0e"
server_key_version: "encryptionkey@0"
I20260618 22:27:36.765154 17908 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260618 22:27:36.768736 18009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:36.770321 17908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260618 22:27:36.770546 17908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/wal
uuid: "bfd01e9042ff4ee3819c16bcee8dd973"
format_stamp: "Formatted at 2026-06-18 22:27:36 on dist-test-slave-0956"
server_key: "2173903d37a9e19be5aa05bbc1645e6f"
server_key_iv: "8fb9d72eb10d3b7e9f8757c6b6b15f0e"
server_key_version: "encryptionkey@0"
I20260618 22:27:36.770735 17908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:36.811094 17908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:36.815171 17908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:36.815434 17908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:36.835234 17908 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.252:35943
I20260618 22:27:36.835263 18060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.252:35943 every 8 connection(s)
I20260618 22:27:36.836583 17908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/data/info.pb
I20260618 22:27:36.840237 18061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:36.847230 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 17908
I20260618 22:27:36.847729 18061 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:27:36 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821656, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.252@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM
Jun 18 22:27:36 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.252@KRBTEST.COM
I20260618 22:27:36.874233 17877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.854647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33978 (local address 127.10.242.254:34635)
0618 22:27:36.854828 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:36.854834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:36.854856 (+ 22us) server_negotiation.cc:407] Connection header received
0618 22:27:36.857184 (+ 2328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:36.857190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:36.857288 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:36.857385 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:36.859286 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.860108 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.862382 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.862991 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.863088 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.867438 (+ 4350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:36.867445 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:36.867450 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:36.867571 (+ 121us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:36.870754 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.871535 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.871541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.871545 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.871691 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.872150 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.872154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.872158 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.872514 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:36.872660 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:36.873721 (+ 1061us) server_negotiation.cc:299] Negotiation successful
0618 22:27:36.873965 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:27:36.875589 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.852614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.0.0.1:33978)
0618 22:27:36.853876 (+ 1262us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:36.853936 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:36.856836 (+ 2900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:36.857496 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:36.857504 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:36.858435 (+ 931us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:36.858932 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.859089 (+ 157us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.860300 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.860305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.862207 (+ 1902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.862227 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.863400 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.863406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.863604 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.864278 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:36.864301 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:36.867167 (+ 2866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:36.870882 (+ 3715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.870894 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.870909 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.871396 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.871800 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.871805 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.871809 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.872043 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.872643 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:36.872652 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:36.873226 (+ 574us) client_negotiation.cc:769] Sending connection context
0618 22:27:36.873574 (+ 348us) client_negotiation.cc:240] Negotiation successful
0618 22:27:36.874139 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"mutex_wait_us":163,"thread_start_us":491,"threads_started":1}
I20260618 22:27:36.877454 18065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.852614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41342 (local address 127.10.242.252:35943)
0618 22:27:36.855358 (+ 2744us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:36.855366 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:36.855413 (+ 47us) server_negotiation.cc:407] Connection header received
0618 22:27:36.856941 (+ 1528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:36.856974 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:36.857329 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:36.857562 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:36.859430 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.861512 (+ 2082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.863295 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.864278 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.864442 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.869566 (+ 5124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:36.869579 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:36.869599 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:36.869666 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:36.874355 (+ 4689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.875345 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.875353 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.875356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.875514 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.875951 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.875956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.875960 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.876263 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:36.876490 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:36.876851 (+ 361us) server_negotiation.cc:299] Negotiation successful
0618 22:27:36.877029 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":80,"server-negotiator.queue_time_us":2517,"thread_start_us":2258,"threads_started":1}
I20260618 22:27:36.878208 18061 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:27:36 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821656, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.252@KRBTEST.COM for oryx/127.10.242.253@KRBTEST.COM
I20260618 22:27:36.890913 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.878989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:33505 (local address 127.0.0.1:60082)
0618 22:27:36.879179 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:36.879200 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:36.879320 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:36.879663 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:36.879666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:36.880109 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:36.880300 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.880309 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.881351 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.881356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.882267 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.882279 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.882960 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.882965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.883114 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.883645 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:36.883665 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:36.885762 (+ 2097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:36.888587 (+ 2825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.888597 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.888600 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.888954 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.889408 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.889412 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.889414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.889549 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.890124 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:36.890129 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:36.890269 (+ 140us) client_negotiation.cc:769] Sending connection context
0618 22:27:36.890392 (+ 123us) client_negotiation.cc:240] Negotiation successful
0618 22:27:36.890648 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260618 22:27:36.891009 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.879006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60082 (local address 127.10.242.253:33505)
0618 22:27:36.879196 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:36.879202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:36.879247 (+ 45us) server_negotiation.cc:407] Connection header received
0618 22:27:36.879427 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:36.879432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:36.879507 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:36.879610 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:36.880425 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.881221 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.882388 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.882798 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.882892 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.885975 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:36.885980 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:36.885983 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:36.886016 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:36.888439 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.889137 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.889145 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.889148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.889299 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.889668 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.889673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.889675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.889975 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:36.890140 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:36.890515 (+ 375us) server_negotiation.cc:299] Negotiation successful
0618 22:27:36.890739 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260618 22:27:36.892138 18061 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:27:36 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821656, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.252@KRBTEST.COM for oryx/127.10.242.252@KRBTEST.COM
I20260618 22:27:36.904477 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.892895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:35943 (local address 127.0.0.1:41356)
0618 22:27:36.893142 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:36.893157 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:36.893251 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:36.893576 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:36.893579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:36.893957 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:36.894148 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.894157 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.895132 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.895136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.896042 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.896053 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.896804 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.896809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.896978 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.897574 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:36.897592 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:36.899587 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:36.902354 (+ 2767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.902358 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.902360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.902717 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.903056 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.903059 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.903062 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.903201 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.903718 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:36.903721 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:36.903843 (+ 122us) client_negotiation.cc:769] Sending connection context
0618 22:27:36.903957 (+ 114us) client_negotiation.cc:240] Negotiation successful
0618 22:27:36.904160 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260618 22:27:36.904556 18065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.892935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41356 (local address 127.10.242.252:35943)
0618 22:27:36.893141 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:36.893145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:36.893213 (+ 68us) server_negotiation.cc:407] Connection header received
0618 22:27:36.893333 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:36.893336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:36.893394 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:36.893524 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:36.894289 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.895005 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.896181 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.896648 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.896753 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.899755 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:36.899760 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:36.899763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:36.899798 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:36.902235 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.902814 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.902818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.902820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.902925 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.903321 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.903325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.903327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.903609 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:36.903718 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:36.904067 (+ 349us) server_negotiation.cc:299] Negotiation successful
0618 22:27:36.904283 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":50}
I20260618 22:27:36.909279 18061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: Bootstrap starting.
I20260618 22:27:36.912263 18061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:36.913676 18061 log.cc:824] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:36.916253 18061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: No bootstrap required, opened a new log
I20260618 22:27:36.923067 18061 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } }
I20260618 22:27:36.923389 18061 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:36.923484 18061 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfd01e9042ff4ee3819c16bcee8dd973, State: Initialized, Role: FOLLOWER
I20260618 22:27:36.924029 18061 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } }
I20260618 22:27:36.924837 18067 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } } }
I20260618 22:27:36.925235 18067 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:27:36.925820 18061 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:27:36.932142 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 17406
W20260618 22:27:36.935966 18078 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:27:36.936075 18078 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:27:36.940061 17908 master_runner.cc:428] Detected that this master bfd01e9042ff4ee3819c16bcee8dd973 is joining an existing cluster
I20260618 22:27:36.940196 17908 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.242.252:35943
Jun 18 22:27:36 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.193@KRBTEST.COM
I20260618 22:27:36.958137 18079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.933190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39420 (local address 127.10.242.193:46667)
0618 22:27:36.936580 (+ 3390us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:36.936589 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:36.936617 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:27:36.936671 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:36.936677 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:36.936917 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:36.937107 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:36.938519 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.940430 (+ 1911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.942083 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.942756 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.942835 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.949930 (+ 7095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:36.949953 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:36.949975 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:36.950035 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:36.954127 (+ 4092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.955157 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.955169 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.955173 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.955356 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.955866 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.955872 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.955876 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.956367 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:36.956678 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:36.957364 (+ 686us) server_negotiation.cc:299] Negotiation successful
0618 22:27:36.957695 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3220,"thread_start_us":3080,"threads_started":1}
I20260618 22:27:36.961050 17529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:36.968506 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.944539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60096 (local address 127.10.242.253:33505)
0618 22:27:36.944723 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:36.944729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:36.951365 (+ 6636us) server_negotiation.cc:407] Connection header received
0618 22:27:36.952246 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:36.952254 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:36.952341 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:36.952411 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:36.955181 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.956219 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.958244 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.959001 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.959083 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.961693 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:36.961700 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:36.961705 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:36.961746 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:36.965069 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.966019 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.966025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.966028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.966187 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.966757 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.966762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.966766 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.967123 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:36.967260 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:36.967858 (+ 598us) server_negotiation.cc:299] Negotiation successful
0618 22:27:36.968121 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:27:36.969624 18085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.944591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:33505 (local address 127.0.0.1:60096)
0618 22:27:36.951250 (+ 6659us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:36.951272 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:36.951907 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:36.952657 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:36.952662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:36.953431 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:36.953782 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.953801 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.956426 (+ 2625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.956432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.958109 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.958121 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.959394 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.959400 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.959604 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.960302 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:36.960324 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:36.961466 (+ 1142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:36.965262 (+ 3796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.965267 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.965275 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.965862 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.966333 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.966338 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.966342 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.966625 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.967334 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:36.967339 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:36.967557 (+ 218us) client_negotiation.cc:769] Sending connection context
0618 22:27:36.968751 (+ 1194us) client_negotiation.cc:240] Negotiation successful
0618 22:27:36.969116 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6497,"mutex_wait_us":23,"spinlock_wait_cycles":28032,"thread_start_us":6569,"threads_started":1}
I20260618 22:27:36.974383 18065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.945168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41360 (local address 127.10.242.252:35943)
0618 22:27:36.945329 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:36.945336 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:36.955246 (+ 9910us) server_negotiation.cc:407] Connection header received
0618 22:27:36.955460 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:36.955466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:36.955528 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:36.955588 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:36.956877 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.957994 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.963417 (+ 5423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.964035 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.964148 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.966612 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:36.966618 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:36.966623 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:36.966655 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:36.971024 (+ 4369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.972073 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.972078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.972081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.972226 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.972614 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.972619 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.972623 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.973061 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:36.973224 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:36.973712 (+ 488us) server_negotiation.cc:299] Negotiation successful
0618 22:27:36.973971 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":37}
I20260618 22:27:36.975370 18086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.945285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:35943 (local address 127.0.0.1:41360)
0618 22:27:36.955162 (+ 9877us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:36.955181 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:36.955326 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:36.955748 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:36.955753 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:36.956369 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:36.956730 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.956741 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.958733 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.958738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.963263 (+ 4525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.963279 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.964437 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.964454 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.964613 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.965247 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:36.965270 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:36.966400 (+ 1130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:36.971436 (+ 5036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.971442 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.971446 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.971936 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.972338 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.972343 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.972347 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.972496 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.973284 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:36.973289 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:36.973463 (+ 174us) client_negotiation.cc:769] Sending connection context
0618 22:27:36.974544 (+ 1081us) client_negotiation.cc:240] Negotiation successful
0618 22:27:36.974820 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9744,"thread_start_us":9528,"threads_started":1}
I20260618 22:27:36.987519 17877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.950936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33980 (local address 127.10.242.254:34635)
0618 22:27:36.951135 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:36.951140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:36.964383 (+ 13243us) server_negotiation.cc:407] Connection header received
0618 22:27:36.964533 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:36.964538 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:36.964620 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:36.964706 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:36.973715 (+ 9009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.974884 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.977638 (+ 2754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.978317 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.978454 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.980854 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:36.980861 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:36.980867 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:36.980906 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:36.984299 (+ 3393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.985146 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.985151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.985155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.985331 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:36.985787 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:36.985793 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:36.985796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:36.986201 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:36.986343 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:36.986904 (+ 561us) server_negotiation.cc:299] Negotiation successful
0618 22:27:36.987164 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260618 22:27:36.987517 18087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.951816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.0.0.1:33980)
0618 22:27:36.964269 (+ 12453us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:36.964284 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:36.964415 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:36.964802 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:36.964807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:36.973155 (+ 8348us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:36.973518 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.973528 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.975364 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.975369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.977455 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.977473 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.978524 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.978531 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.978712 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:36.979443 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:36.979465 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:36.980583 (+ 1118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:36.984479 (+ 3896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.984485 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.984489 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.984977 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.985448 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:36.985453 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:36.985457 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:36.985634 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:36.986325 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:36.986330 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:36.986582 (+ 252us) client_negotiation.cc:769] Sending connection context
0618 22:27:36.986735 (+ 153us) client_negotiation.cc:240] Negotiation successful
0618 22:27:36.987038 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12321,"thread_start_us":12659,"threads_started":1}
Jun 18 22:27:36 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821654, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.254@KRBTEST.COM for oryx/127.10.242.252@KRBTEST.COM
I20260618 22:27:37.010512 17900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.993849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:35943 (local address 127.0.0.1:41376)
0618 22:27:36.994035 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:36.994055 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:36.994160 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:36.995190 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:36.995195 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:36.995661 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:36.995867 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.995880 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.997657 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.997663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:36.999034 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:36.999051 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.000119 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.000126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:37.000299 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:37.000982 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:37.001007 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:37.003773 (+ 2766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:37.007794 (+ 4021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:37.007800 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:37.007805 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:37.008287 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:37.008677 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:37.008682 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:37.008686 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:37.008844 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:37.009498 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:37.009503 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:37.009690 (+ 187us) client_negotiation.cc:769] Sending connection context
0618 22:27:37.009830 (+ 140us) client_negotiation.cc:240] Negotiation successful
0618 22:27:37.010118 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260618 22:27:37.010854 18065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:36.993849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41376 (local address 127.10.242.252:35943)
0618 22:27:36.994369 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:36.994374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:36.994391 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:27:36.994918 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:36.994925 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:36.995028 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:36.996069 (+ 1041us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:36.996283 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.997480 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:36.999227 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:36.999928 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.001237 (+ 1309us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:37.004005 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:37.004015 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:37.004021 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:37.004061 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:37.007617 (+ 3556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:37.008420 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:37.008426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:37.008429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:37.008571 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:37.008959 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:37.008964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:37.008967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:37.009384 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:37.009537 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:37.009957 (+ 420us) server_negotiation.cc:299] Negotiation successful
0618 22:27:37.010185 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404}
I20260618 22:27:37.012578 17345 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.242.252:35943: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973"
member_type: NON_VOTER
last_known_addr {
host: "127.10.242.252"
port: 35943
}
attrs {
promote: true
}
}
dest_uuid: "7a255bfe590d478991390a7b3b7e9b80"
cas_config_opid_index: 5
I20260618 22:27:37.013700 17345 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: NON_VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: true } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: NON_VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: true } }
I20260618 22:27:37.015249 17904 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER bfd01e9042ff4ee3819c16bcee8dd973 (127.10.242.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: NON_VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: true } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: NON_VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: true } } }
I20260618 22:27:37.016315 18036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:37.020171 18036 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 1 FOLLOWER]: Refusing update from remote peer 7a255bfe590d478991390a7b3b7e9b80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260618 22:27:37.030843 17904 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.242.252:35943
I20260618 22:27:37.031561 17908 master.cc:537] Master@127.10.242.252:35943 shutting down...
I20260618 22:27:37.032173 17954 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 1 LEARNER]: Refusing update from remote peer 7a255bfe590d478991390a7b3b7e9b80: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260618 22:27:37.033208 18091 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: NON_VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.001s
I20260618 22:27:37.037467 18089 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: NON_VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260618 22:27:37.043660 17954 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER bfd01e9042ff4ee3819c16bcee8dd973 (127.10.242.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: NON_VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: true } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: NON_VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: true } } }
I20260618 22:27:37.045253 18090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7a255bfe590d478991390a7b3b7e9b80" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: NON_VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: true } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: NON_VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: true } } }
I20260618 22:27:37.045632 18090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: This master's current role is: LEADER
W20260618 22:27:37.049394 17329 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 -> Peer bfd01e9042ff4ee3819c16bcee8dd973 (127.10.242.252:35943): Couldn't send request to peer bfd01e9042ff4ee3819c16bcee8dd973. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260618 22:27:37.050055 17982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7a255bfe590d478991390a7b3b7e9b80" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: NON_VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: true } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: NON_VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: true } } }
I20260618 22:27:37.050411 17982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [sys.catalog]: This master's current role is: LEARNER
I20260618 22:27:37.051683 18091 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80: attempting to promote NON_VOTER 468d5c19d3c54b7bbd0d7122ea722111 to VOTER
I20260618 22:27:37.052726 18091 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: NON_VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: true } }
I20260618 22:27:37.055276 17954 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 1 LEARNER]: Refusing update from remote peer 7a255bfe590d478991390a7b3b7e9b80: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260618 22:27:37.055783 18091 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260618 22:27:37.055981 17329 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 -> Peer bfd01e9042ff4ee3819c16bcee8dd973 (127.10.242.252:35943): Couldn't send request to peer bfd01e9042ff4ee3819c16bcee8dd973. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260618 22:27:37.058033 18091 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 468d5c19d3c54b7bbd0d7122ea722111 (127.10.242.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: NON_VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: true } } }
I20260618 22:27:37.061829 18089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7a255bfe590d478991390a7b3b7e9b80" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: NON_VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: true } } }
I20260618 22:27:37.062182 18089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:37.062497 18089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7a255bfe590d478991390a7b3b7e9b80" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: NON_VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: true } } }
I20260618 22:27:37.062703 18089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:37.063799 17953 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 468d5c19d3c54b7bbd0d7122ea722111 (127.10.242.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: NON_VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: true } } }
I20260618 22:27:37.067528 17908 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:37.067962 17908 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:37.068169 17908 tablet_replica.cc:330] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: stopping tablet replica
I20260618 22:27:37.068058 17982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7a255bfe590d478991390a7b3b7e9b80" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: NON_VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: true } } }
I20260618 22:27:37.068302 17982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:27:37.074759 17908 master.cc:552] Master@127.10.242.252:35943 shutdown complete.
I20260618 22:27:37.074849 17908 master_runner.cc:305] Clearing existing system tablet
I20260618 22:27:37.075278 17908 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:37.075393 17908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:37.078403 17908 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260618 22:27:37.079936 18096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:37.080323 17908 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:37.080673 17908 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260618 22:27:37.080801 17908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/wal
uuid: "bfd01e9042ff4ee3819c16bcee8dd973"
format_stamp: "Formatted at 2026-06-18 22:27:36 on dist-test-slave-0956"
server_key: "2173903d37a9e19be5aa05bbc1645e6f"
server_key_iv: "8fb9d72eb10d3b7e9f8757c6b6b15f0e"
server_key_version: "encryptionkey@0"
I20260618 22:27:37.087620 17908 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:37.088799 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 17406
I20260618 22:27:37.097669 17908 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260618 22:27:37.097859 17908 log.cc:1196] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260618 22:27:37.098330 17908 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: Deleting consensus metadata
I20260618 22:27:37.098593 17908 master_runner.cc:315] Copying system tablet from 127.10.242.254:34635
I20260618 22:27:37.107719 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:46667
--local_ip_for_outbound_sockets=127.10.242.193
--tserver_master_addrs=127.10.242.254:34635
--webserver_port=34705
--webserver_interface=127.10.242.193
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.193.keytab
--principal=oryx/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.242.254:34635,127.10.242.253:33505
--tserver_master_addrs=127.10.242.254:34635,127.10.242.253:33505,127.10.242.252:35943 with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:27:37.108579 17908 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: tablet copy: Beginning tablet copy session from remote peer at address 127.10.242.254:34635
I20260618 22:27:37.126719 17877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:37.109484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33990 (local address 127.10.242.254:34635)
0618 22:27:37.112745 (+ 3261us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:37.112753 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:37.112776 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:27:37.112842 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:37.112847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:37.112926 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:37.113048 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:37.114159 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.115266 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.117028 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.117707 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.117839 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:37.120004 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:37.120011 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:37.120016 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:37.120056 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:37.123736 (+ 3680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:37.124562 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:37.124568 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:37.124572 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:37.124727 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:37.125146 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:37.125151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:37.125154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:37.125495 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:37.125631 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:37.126133 (+ 502us) server_negotiation.cc:299] Negotiation successful
0618 22:27:37.126357 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3114}
I20260618 22:27:37.127260 18102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:37.109471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.0.0.1:33990)
0618 22:27:37.109967 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:37.110002 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:37.110145 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:37.113150 (+ 3005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:37.113155 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:37.113669 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:37.113959 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:37.113973 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.115435 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.115440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:37.116848 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:37.116869 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.117883 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.117893 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:37.118065 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:37.118765 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:37.118786 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:37.119792 (+ 1006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:37.123897 (+ 4105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:37.123903 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:37.123908 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:37.124414 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:37.124840 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:37.124846 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:37.124849 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:37.125025 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:37.125669 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:37.125674 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:37.125869 (+ 195us) client_negotiation.cc:769] Sending connection context
0618 22:27:37.126029 (+ 160us) client_negotiation.cc:240] Negotiation successful
0618 22:27:37.126409 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":233,"threads_started":1}
I20260618 22:27:37.127911 17365 tablet_copy_service.cc:140] P 7a255bfe590d478991390a7b3b7e9b80: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bfd01e9042ff4ee3819c16bcee8dd973 ({username='oryx', principal='oryx/127.10.242.252@KRBTEST.COM'} at 127.0.0.1:33990)
I20260618 22:27:37.128089 17365 tablet_copy_service.cc:161] P 7a255bfe590d478991390a7b3b7e9b80: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bfd01e9042ff4ee3819c16bcee8dd973 at {username='oryx', principal='oryx/127.10.242.252@KRBTEST.COM'} at 127.0.0.1:33990: session id = bfd01e9042ff4ee3819c16bcee8dd973-00000000000000000000000000000000
I20260618 22:27:37.131763 17365 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80: Tablet Copy: opened 0 blocks and 1 log segments
I20260618 22:27:37.133704 17908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260618 22:27:37.140710 17908 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: tablet copy: Starting download of 0 data blocks...
I20260618 22:27:37.141026 17908 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: tablet copy: Starting download of 1 WAL segments...
I20260618 22:27:37.143639 17908 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260618 22:27:37.148074 17365 tablet_copy_service.cc:342] P 7a255bfe590d478991390a7b3b7e9b80: Request end of tablet copy session bfd01e9042ff4ee3819c16bcee8dd973-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.242.252@KRBTEST.COM'} at 127.0.0.1:33990
I20260618 22:27:37.148221 17365 tablet_copy_service.cc:433] P 7a255bfe590d478991390a7b3b7e9b80: ending tablet copy session bfd01e9042ff4ee3819c16bcee8dd973-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bfd01e9042ff4ee3819c16bcee8dd973
I20260618 22:27:37.159690 17908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:37.162511 18104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:37.163036 18105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:37.173002 18107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:37.178817 17908 server_base.cc:1034] running on GCE node
I20260618 22:27:37.179246 17908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:37.179911 17908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:37.181145 17908 hybrid_clock.cc:648] HybridClock initialized: now 1781821657181096 us; error 43 us; skew 500 ppm
I20260618 22:27:37.181473 17908 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:37 dist-test-slave-0956 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821657, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:37.183796 17908 init.cc:377] Logged in from keytab as oryx/127.10.242.252@KRBTEST.COM (short username oryx)
I20260618 22:27:37.186059 17908 webserver.cc:460] Webserver started at http://127.10.242.252:46439/ using document root <none> and password file <none>
I20260618 22:27:37.186389 17908 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:37.186481 17908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:37.189590 17908 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20260618 22:27:37.191592 18114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:37.191988 17908 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260618 22:27:37.192114 17908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/wal
uuid: "bfd01e9042ff4ee3819c16bcee8dd973"
format_stamp: "Formatted at 2026-06-18 22:27:36 on dist-test-slave-0956"
server_key: "2173903d37a9e19be5aa05bbc1645e6f"
server_key_iv: "8fb9d72eb10d3b7e9f8757c6b6b15f0e"
server_key_version: "encryptionkey@0"
I20260618 22:27:37.192291 17908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:37.228875 17908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:37.233011 17908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:37.257359 17908 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.252:35943
I20260618 22:27:37.257524 18165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.252:35943 every 8 connection(s)
I20260618 22:27:37.258383 17908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/master-2/data/info.pb
I20260618 22:27:37.263674 18166 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:27:37.267632 18166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: Bootstrap starting.
I20260618 22:27:37.286629 18166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:27:37.287218 18166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: Bootstrap complete.
I20260618 22:27:37.288352 18166 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: NON_VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: true } }
I20260618 22:27:37.288657 18166 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bfd01e9042ff4ee3819c16bcee8dd973, State: Initialized, Role: LEARNER
I20260618 22:27:37.288924 18166 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: NON_VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: true } }
I20260618 22:27:37.290117 18166 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:27:37.289865 18169 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: NON_VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: true } } }
I20260618 22:27:37.290410 18169 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [sys.catalog]: This master's current role is: LEARNER
I20260618 22:27:37.300028 18180 catalog_manager.cc:1259] Loaded cluster ID: c4e1d5b328594abba209eac27d127622
I20260618 22:27:37.300155 18180 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: loading cluster ID for follower catalog manager: success
I20260618 22:27:37.303704 18180 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: acquiring CA information for follower catalog manager: success
I20260618 22:27:37.306486 18180 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260618 22:27:37.472360 18103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:37.472766 18103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:37.473065 18103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:37.484215 18103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:37.484356 18103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:37.484552 18103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:27:37.500880 18103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.193.keytab
--principal=oryx/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:46667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=34705
--tserver_master_addrs=127.10.242.254:34635,127.10.242.253:33505,127.10.242.252:35943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:37.502632 18103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:37.504338 18103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:37.519235 18186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:37.519361 18185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:37.519433 18188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:37.520992 18103 server_base.cc:1034] running on GCE node
I20260618 22:27:37.521595 18103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:37.522538 18103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:37.523751 18103 hybrid_clock.cc:648] HybridClock initialized: now 1781821657523693 us; error 51 us; skew 500 ppm
I20260618 22:27:37.524056 18103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:37 dist-test-slave-0956 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821657, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:37.527190 18103 init.cc:377] Logged in from keytab as oryx/127.10.242.193@KRBTEST.COM (short username oryx)
I20260618 22:27:37.528729 18103 webserver.cc:460] Webserver started at http://127.10.242.193:34705/ using document root <none> and password file <none>
I20260618 22:27:37.529500 18103 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:37.529623 18103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:37.535027 18103 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:27:37.538331 18195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:37.540081 18103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:27:37.540256 18103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "5899712e99ff4d88bb415b836db9967c"
format_stamp: "Formatted at 2026-06-18 22:27:34 on dist-test-slave-0956"
server_key: "99dd079ae1f3ef21e8aa2510debf6c0c"
server_key_iv: "d9d755156c41140d06854032b0fce6e3"
server_key_version: "encryptionkey@0"
I20260618 22:27:37.541354 18103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:37.555238 18103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:37.559190 18103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:37.559492 18103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:37.560457 18103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:37.561849 18103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:37.561939 18103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:37.562039 18103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:37.562104 18103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:37.600162 18103 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:46667
I20260618 22:27:37.600178 18307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:46667 every 8 connection(s)
I20260618 22:27:37.602115 18103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:27:37.610831 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 18103
I20260618 22:27:37.611157 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 17540
Jun 18 22:27:37 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821657, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.252@KRBTEST.COM
I20260618 22:27:37.711119 18312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:37.605334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:53069 (local address 127.10.242.252:35943)
0618 22:27:37.664696 (+ 59362us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:37.664702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:37.690136 (+ 25434us) server_negotiation.cc:407] Connection header received
0618 22:27:37.690308 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:37.690313 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:37.690423 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:37.690558 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:37.694078 (+ 3520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.695473 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.697979 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.698588 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.698706 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:37.704067 (+ 5361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:37.704075 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:37.704080 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:37.704122 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:37.707323 (+ 3201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:37.708332 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:37.708337 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:37.708340 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:37.708484 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:37.709004 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:37.709009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:37.709012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:37.709401 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:37.709525 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:37.710523 (+ 998us) server_negotiation.cc:299] Negotiation successful
0618 22:27:37.710740 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59234,"thread_start_us":59121,"threads_started":1}
I20260618 22:27:37.713785 18316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:37.606735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:35943 (local address 127.10.242.193:53069)
0618 22:27:37.689996 (+ 83261us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:37.690016 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:37.690179 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:37.691162 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:37.691176 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:37.692549 (+ 1373us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:37.693775 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:37.693785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.695693 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.695699 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:37.697822 (+ 2123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:37.697842 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.699088 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.699096 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:37.699364 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:37.700109 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:37.700141 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:37.703806 (+ 3665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:37.707498 (+ 3692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:37.707507 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:37.707525 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:37.708189 (+ 664us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:37.708619 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:37.708623 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:37.708627 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:37.708860 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:37.709609 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:37.709617 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:37.710050 (+ 433us) client_negotiation.cc:769] Sending connection context
0618 22:27:37.711644 (+ 1594us) client_negotiation.cc:240] Negotiation successful
0618 22:27:37.712248 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83126,"thread_start_us":82971,"threads_started":1}
I20260618 22:27:37.715965 18308 heartbeater.cc:347] Connected to a master server at 127.10.242.252:35943
I20260618 22:27:37.716331 18308 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:37.717312 18308 heartbeater.cc:511] Master 127.10.242.252:35943 requested a full tablet report, sending...
I20260618 22:27:37.718456 18319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:37.668147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:35943 (local address 127.0.0.1:41380)
0618 22:27:37.695979 (+ 27832us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:37.696001 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:37.696121 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:37.698015 (+ 1894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:37.698020 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:37.698051 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:37.698209 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:37.698219 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.699977 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.699988 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:37.701854 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:37.701873 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.714986 (+ 13113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.714992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:37.717857 (+ 2865us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:37.717897 (+ 40us) client_negotiation.cc:769] Sending connection context
0618 22:27:37.717996 (+ 99us) client_negotiation.cc:240] Negotiation successful
0618 22:27:37.718149 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27696,"thread_start_us":28790,"threads_started":1}
I20260618 22:27:37.720240 18131 ts_manager.cc:194] Registered new tserver with Master: 5899712e99ff4d88bb415b836db9967c (127.10.242.193:46667)
I20260618 22:27:37.722379 18320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:37.668629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41380 (local address 127.10.242.252:35943)
0618 22:27:37.696528 (+ 27899us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:37.696533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:37.696549 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:27:37.696604 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:37.696609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:37.696689 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:37.696796 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:37.698363 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.699527 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.709367 (+ 9840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.714446 (+ 5079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.714658 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:37.714853 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:37.721905 (+ 7052us) server_negotiation.cc:299] Negotiation successful
0618 22:27:37.722068 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27767,"thread_start_us":28770,"threads_started":1}
Jun 18 22:27:37 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.194@KRBTEST.COM
I20260618 22:27:37.723182 18141 raft_consensus.cc:1203] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260618 22:27:37.723963 18169 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a255bfe590d478991390a7b3b7e9b80. Latest consensus state: current_term: 1 leader_uuid: "7a255bfe590d478991390a7b3b7e9b80" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: NON_VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: true } } }
I20260618 22:27:37.724226 18169 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [sys.catalog]: This master's current role is: LEARNER
Jun 18 22:27:37 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821657, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.253@KRBTEST.COM
Jun 18 22:27:37 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821657, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM
I20260618 22:27:37.733880 18318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:37.612552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49682 (local address 127.10.242.194:42531)
0618 22:27:37.654521 (+ 41969us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:37.654532 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:37.654566 (+ 34us) server_negotiation.cc:407] Connection header received
0618 22:27:37.654646 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:37.654657 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:37.654965 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:37.655131 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:37.664810 (+ 9679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.697826 (+ 33016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.707423 (+ 9597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.708356 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.708470 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:37.723897 (+ 15427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:37.723918 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:37.723936 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:37.723984 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:37.728886 (+ 4902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:37.729968 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:37.729978 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:37.729982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:37.730158 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:37.731305 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:37.731309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:37.731311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:37.731747 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:37.732051 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:37.733202 (+ 1151us) server_negotiation.cc:299] Negotiation successful
0618 22:27:37.733465 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41760,"thread_start_us":47052,"threads_started":1}
I20260618 22:27:37.735450 17663 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:37.737968 18313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:37.606443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:41177 (local address 127.10.242.253:33505)
0618 22:27:37.661247 (+ 54804us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:37.661254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:37.667143 (+ 5889us) server_negotiation.cc:407] Connection header received
0618 22:27:37.667219 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:37.667225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:37.690370 (+ 23145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:37.690485 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:37.693980 (+ 3495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.695241 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.703383 (+ 8142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.704016 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.704125 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:37.729337 (+ 25212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:37.729347 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:37.729352 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:37.729395 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:37.733047 (+ 3652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:37.733907 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:37.733913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:37.733917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:37.735198 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:37.735719 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:37.735725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:37.735729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:37.736131 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:37.736294 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:37.737274 (+ 980us) server_negotiation.cc:299] Negotiation successful
0618 22:27:37.737544 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54663,"thread_start_us":80250,"threads_started":1}
I20260618 22:27:37.738852 18314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:37.606615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:33505 (local address 127.10.242.193:41177)
0618 22:27:37.666720 (+ 60105us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:37.666742 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:37.666887 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:37.692626 (+ 25739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:37.692630 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:37.693262 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:37.693750 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:37.693778 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.702049 (+ 8271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.702054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:37.703182 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:37.703198 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.710098 (+ 6900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.710104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:37.710301 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:37.715390 (+ 5089us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:37.715415 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:37.728984 (+ 13569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:37.733230 (+ 4246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:37.733235 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:37.733239 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:37.733750 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:37.735349 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:37.735355 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:37.735359 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:37.735526 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:37.736361 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:37.736366 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:37.736554 (+ 188us) client_negotiation.cc:769] Sending connection context
0618 22:27:37.736692 (+ 138us) client_negotiation.cc:240] Negotiation successful
0618 22:27:37.737027 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59967,"thread_start_us":53204,"threads_started":1}
I20260618 22:27:37.739996 18310 heartbeater.cc:347] Connected to a master server at 127.10.242.253:33505
I20260618 22:27:37.740161 18310 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:37.740190 17877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:37.604992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:41795 (local address 127.10.242.254:34635)
0618 22:27:37.605199 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:37.605205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:37.656619 (+ 51414us) server_negotiation.cc:407] Connection header received
0618 22:27:37.705044 (+ 48425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:37.705050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:37.705188 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:37.705264 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:37.710773 (+ 5509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.711877 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.725671 (+ 13794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.726362 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.726468 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:37.732836 (+ 6368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:37.732844 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:37.732849 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:37.732891 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:37.736497 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:37.737627 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:37.737633 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:37.737638 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:37.737786 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:37.738252 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:37.738257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:37.738261 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:37.738648 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:37.738825 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:37.739580 (+ 755us) server_negotiation.cc:299] Negotiation successful
0618 22:27:37.739836 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260618 22:27:37.740688 18310 heartbeater.cc:511] Master 127.10.242.253:33505 requested a full tablet report, sending...
I20260618 22:27:37.741159 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:37.606292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.193:41795)
0618 22:27:37.656419 (+ 50127us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:37.656483 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:37.659232 (+ 2749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:37.705462 (+ 46230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:37.705467 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:37.708873 (+ 3406us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:37.709194 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:37.709207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.712098 (+ 2891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.712103 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:37.713615 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:37.713650 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:37.726839 (+ 13189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:37.726848 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:37.727064 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:37.729010 (+ 1946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:37.729037 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:37.732573 (+ 3536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:37.736978 (+ 4405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:37.736983 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:37.736988 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:37.737475 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:37.737932 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:37.737951 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:37.737955 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:37.738133 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:37.738944 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:37.738949 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:37.739225 (+ 276us) client_negotiation.cc:769] Sending connection context
0618 22:27:37.740368 (+ 1143us) client_negotiation.cc:240] Negotiation successful
0618 22:27:37.740717 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49352,"spinlock_wait_cycles":314624,"thread_start_us":60011,"threads_started":1}
I20260618 22:27:37.743316 17944 ts_manager.cc:194] Registered new tserver with Master: 5899712e99ff4d88bb415b836db9967c (127.10.242.193:46667)
I20260618 22:27:37.743891 18309 heartbeater.cc:347] Connected to a master server at 127.10.242.254:34635
I20260618 22:27:37.744047 18309 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:37.744530 18309 heartbeater.cc:511] Master 127.10.242.254:34635 requested a full tablet report, sending...
I20260618 22:27:37.745963 17345 ts_manager.cc:194] Re-registered known tserver with Master: 5899712e99ff4d88bb415b836db9967c (127.10.242.193:46667)
I20260618 22:27:37.747220 17345 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:41795
I20260618 22:27:37.832706 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 17540
I20260618 22:27:37.845757 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:42531
--local_ip_for_outbound_sockets=127.10.242.194
--tserver_master_addrs=127.10.242.254:34635
--webserver_port=40421
--webserver_interface=127.10.242.194
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.194.keytab
--principal=oryx/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.242.254:34635,127.10.242.253:33505
--tserver_master_addrs=127.10.242.254:34635,127.10.242.253:33505,127.10.242.252:35943 with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:27:38.092909 18090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7a255bfe590d478991390a7b3b7e9b80" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: NON_VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: true } } }
I20260618 22:27:38.093277 18090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:38.094970 18323 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80: attempting to promote NON_VOTER bfd01e9042ff4ee3819c16bcee8dd973 to VOTER
I20260618 22:27:38.095505 18323 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false } }
I20260618 22:27:38.097564 18141 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 1 LEARNER]: Refusing update from remote peer 7a255bfe590d478991390a7b3b7e9b80: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260618 22:27:38.097605 17953 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 1 FOLLOWER]: Refusing update from remote peer 7a255bfe590d478991390a7b3b7e9b80: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260618 22:27:38.098193 18323 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260618 22:27:38.098609 18324 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260618 22:27:38.101822 18324 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, bfd01e9042ff4ee3819c16bcee8dd973 (127.10.242.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false } } }
I20260618 22:27:38.102452 17953 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, bfd01e9042ff4ee3819c16bcee8dd973 (127.10.242.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false } } }
I20260618 22:27:38.105800 18090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7a255bfe590d478991390a7b3b7e9b80" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false } } }
I20260618 22:27:38.106097 18090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:38.106304 18090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7a255bfe590d478991390a7b3b7e9b80" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false } } }
I20260618 22:27:38.106513 18090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:38.115263 18330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7a255bfe590d478991390a7b3b7e9b80" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false } } }
I20260618 22:27:38.115602 18330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:27:38.117599 18324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7a255bfe590d478991390a7b3b7e9b80" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false } } }
I20260618 22:27:38.117935 18324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:38.118428 18141 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, bfd01e9042ff4ee3819c16bcee8dd973 (127.10.242.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false } } }
I20260618 22:27:38.121296 18169 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7a255bfe590d478991390a7b3b7e9b80" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false } } }
I20260618 22:27:38.121544 18169 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [sys.catalog]: This master's current role is: FOLLOWER
W20260618 22:27:38.171241 18322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:38.171548 18322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:38.171639 18322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:38.178962 18322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:38.179085 18322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:38.179203 18322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:27:38.189266 18322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.194.keytab
--principal=oryx/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:42531
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=40421
--tserver_master_addrs=127.10.242.254:34635,127.10.242.253:33505,127.10.242.252:35943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:38.190627 18322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:38.191931 18322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:38.205214 18335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:38.205255 18336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:38.205205 18338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:38.206290 18322 server_base.cc:1034] running on GCE node
I20260618 22:27:38.206914 18322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:38.207855 18322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:38.209059 18322 hybrid_clock.cc:648] HybridClock initialized: now 1781821658209005 us; error 43 us; skew 500 ppm
I20260618 22:27:38.209429 18322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:38 dist-test-slave-0956 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821658, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:38.212406 18322 init.cc:377] Logged in from keytab as oryx/127.10.242.194@KRBTEST.COM (short username oryx)
I20260618 22:27:38.213871 18322 webserver.cc:460] Webserver started at http://127.10.242.194:40421/ using document root <none> and password file <none>
I20260618 22:27:38.214576 18322 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:38.214709 18322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:38.220073 18322 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:27:38.223101 18345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:38.224577 18322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:27:38.224753 18322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "0fb54f92e5ac4441bed6f5bb6ea32144"
format_stamp: "Formatted at 2026-06-18 22:27:35 on dist-test-slave-0956"
server_key: "8a8997b1e0f78f0bfa443f3788b37f91"
server_key_iv: "2b5f4f3e0ed5fb4aeec698ece7e69ed0"
server_key_version: "encryptionkey@0"
I20260618 22:27:38.225831 18322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:38.238737 18322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:38.242929 18322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:38.243192 18322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:38.244196 18322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:38.245605 18322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:38.245703 18322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:38.245802 18322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:38.245869 18322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:38.275178 18322 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:42531
I20260618 22:27:38.275233 18457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:42531 every 8 connection(s)
I20260618 22:27:38.276494 18322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:27:38.284778 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 18322
I20260618 22:27:38.285036 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 17673
Jun 18 22:27:38 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.195@KRBTEST.COM
Jun 18 22:27:38 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821658, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.252@KRBTEST.COM
Jun 18 22:27:38 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821658, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.253@KRBTEST.COM
Jun 18 22:27:38 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821658, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM
I20260618 22:27:38.346788 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.286462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34112 (local address 127.10.242.195:46165)
0618 22:27:38.314957 (+ 28495us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:38.314968 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:38.315006 (+ 38us) server_negotiation.cc:407] Connection header received
0618 22:27:38.315092 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:38.315101 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:38.315526 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:38.315714 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:38.321209 (+ 5495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.322924 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.329064 (+ 6140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.330266 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.330428 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.336255 (+ 5827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:38.336272 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:38.336288 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:38.336832 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:38.342435 (+ 5603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.343299 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.343310 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.343313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.343470 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.343924 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.343929 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.343932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.344333 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:38.345291 (+ 958us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:38.345464 (+ 173us) server_negotiation.cc:299] Negotiation successful
0618 22:27:38.346372 (+ 908us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28243,"thread_start_us":29744,"threads_started":1}
I20260618 22:27:38.347837 18467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.280580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:38549 (local address 127.10.242.252:35943)
0618 22:27:38.326636 (+ 46056us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:38.326794 (+ 158us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:38.326821 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:27:38.328378 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:38.328385 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:38.328485 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:38.328601 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:38.331276 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.332820 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.334722 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.335345 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.335450 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.340434 (+ 4984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:38.340440 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:38.340445 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:38.340494 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:38.343730 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.344853 (+ 1123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.344858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.344862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.344986 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.345572 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.345577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.345581 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.345919 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:38.346061 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:38.347273 (+ 1212us) server_negotiation.cc:299] Negotiation successful
0618 22:27:38.347486 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45931,"thread_start_us":45778,"threads_started":1}
I20260618 22:27:38.348366 18462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.279390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:48561 (local address 127.10.242.254:34635)
0618 22:27:38.279776 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:38.279782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:38.326210 (+ 46428us) server_negotiation.cc:407] Connection header received
0618 22:27:38.328280 (+ 2070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:38.328286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:38.328364 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:38.328442 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:38.330964 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.332005 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.334719 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.335336 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.335447 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.341174 (+ 5727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:38.341180 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:38.341185 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:38.341228 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:38.344815 (+ 3587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.345653 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.345659 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.345662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.345807 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.346219 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.346224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.346227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.346543 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:38.346671 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:38.347582 (+ 911us) server_negotiation.cc:299] Negotiation successful
0618 22:27:38.347781 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":627,"threads_started":1}
I20260618 22:27:38.348776 17796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:38.348766 18465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.279962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:33505 (local address 127.10.242.194:40535)
0618 22:27:38.323534 (+ 43572us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:38.323556 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:38.327911 (+ 4355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:38.331273 (+ 3362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:38.331277 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:38.332116 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:38.332486 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:38.332498 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.333974 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.333979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:38.335412 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:38.335423 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.336323 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.336330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:38.336473 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.336955 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:38.336974 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:38.340433 (+ 3459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:38.343890 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:38.343899 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:38.343918 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:38.344684 (+ 766us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:38.345122 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:38.345127 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:38.345130 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:38.345388 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:38.346138 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:38.346142 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:38.346619 (+ 477us) client_negotiation.cc:769] Sending connection context
0618 22:27:38.346946 (+ 327us) client_negotiation.cc:240] Negotiation successful
0618 22:27:38.347458 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43393,"mutex_wait_us":75,"spinlock_wait_cycles":11008,"thread_start_us":43512,"threads_started":1}
I20260618 22:27:38.349539 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.279999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.194:48561)
0618 22:27:38.326104 (+ 46105us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:38.326122 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:38.327913 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:38.328887 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:38.328901 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:38.329890 (+ 989us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:38.330755 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:38.330780 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.332232 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.332237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:38.334550 (+ 2313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:38.334581 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.336019 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.336025 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:38.336205 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.336792 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:38.336816 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:38.340944 (+ 4128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:38.344965 (+ 4021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:38.344970 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:38.344973 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:38.345530 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:38.345944 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:38.345948 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:38.345952 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:38.346090 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:38.346780 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:38.346785 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:38.347246 (+ 461us) client_negotiation.cc:769] Sending connection context
0618 22:27:38.349010 (+ 1764us) client_negotiation.cc:240] Negotiation successful
0618 22:27:38.349282 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45995,"mutex_wait_us":224,"spinlock_wait_cycles":62208,"thread_start_us":45700,"threads_started":1}
I20260618 22:27:38.350173 18464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.280212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:40535 (local address 127.10.242.253:33505)
0618 22:27:38.330936 (+ 50724us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:38.330943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:38.330961 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:27:38.331028 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:38.331033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:38.331114 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:38.332634 (+ 1520us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:38.332771 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.333815 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.335587 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.336123 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.336736 (+ 613us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.340650 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:38.340656 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:38.340661 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:38.340704 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:38.343765 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.344813 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.344819 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.344822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.344972 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.345573 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.345577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.345581 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.345963 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:38.346083 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:38.349536 (+ 3453us) server_negotiation.cc:299] Negotiation successful
0618 22:27:38.349787 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50608,"thread_start_us":50519,"threads_started":1}
I20260618 22:27:38.350672 18459 heartbeater.cc:347] Connected to a master server at 127.10.242.254:34635
I20260618 22:27:38.350914 18460 heartbeater.cc:347] Connected to a master server at 127.10.242.253:33505
I20260618 22:27:38.350991 18459 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:38.351051 18460 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:38.351851 18459 heartbeater.cc:511] Master 127.10.242.254:34635 requested a full tablet report, sending...
I20260618 22:27:38.351873 18460 heartbeater.cc:511] Master 127.10.242.253:33505 requested a full tablet report, sending...
I20260618 22:27:38.353330 17345 ts_manager.cc:194] Re-registered known tserver with Master: 0fb54f92e5ac4441bed6f5bb6ea32144 (127.10.242.194:42531)
I20260618 22:27:38.353538 17944 ts_manager.cc:194] Registered new tserver with Master: 0fb54f92e5ac4441bed6f5bb6ea32144 (127.10.242.194:42531)
I20260618 22:27:38.354383 17345 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:48561
I20260618 22:27:38.354727 18463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.279962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:35943 (local address 127.10.242.194:38549)
0618 22:27:38.319373 (+ 39411us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:38.319441 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:38.327906 (+ 8465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:38.328887 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:38.328901 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:38.329890 (+ 989us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:38.331113 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:38.331127 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.333038 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.333043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:38.334554 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:38.334580 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.335708 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.335714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:38.336003 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.336678 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:38.336706 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:38.340170 (+ 3464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:38.343890 (+ 3720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:38.343899 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:38.343918 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:38.344679 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:38.345122 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:38.345126 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:38.345130 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:38.345404 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:38.346128 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:38.346135 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:38.346619 (+ 484us) client_negotiation.cc:769] Sending connection context
0618 22:27:38.346930 (+ 311us) client_negotiation.cc:240] Negotiation successful
0618 22:27:38.351035 (+ 4105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39015,"mutex_wait_us":3560,"spinlock_wait_cycles":21632,"thread_start_us":38746,"threads_started":1}
I20260618 22:27:38.355616 18458 heartbeater.cc:347] Connected to a master server at 127.10.242.252:35943
I20260618 22:27:38.355736 18458 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:38.356056 18458 heartbeater.cc:511] Master 127.10.242.252:35943 requested a full tablet report, sending...
I20260618 22:27:38.356961 18131 ts_manager.cc:194] Registered new tserver with Master: 0fb54f92e5ac4441bed6f5bb6ea32144 (127.10.242.194:42531)
I20260618 22:27:38.433530 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 17673
I20260618 22:27:38.446615 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:46165
--local_ip_for_outbound_sockets=127.10.242.195
--tserver_master_addrs=127.10.242.254:34635
--webserver_port=34349
--webserver_interface=127.10.242.195
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.195.keytab
--principal=oryx/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.242.254:34635,127.10.242.253:33505
--tserver_master_addrs=127.10.242.254:34635,127.10.242.253:33505,127.10.242.252:35943 with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:27:38.750519 18309 heartbeater.cc:503] Master 127.10.242.254:34635 was elected leader, sending a full tablet report...
W20260618 22:27:38.763347 18471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:38.763664 18471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:38.763788 18471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:38.771064 18471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:38.771193 18471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:38.771349 18471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:27:38.781127 18471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.195.keytab
--principal=oryx/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:46165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=34349
--tserver_master_addrs=127.10.242.254:34635,127.10.242.253:33505,127.10.242.252:35943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:38.782441 18471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:38.783711 18471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:38.796729 18476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:38.796756 18479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:38.796698 18477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:38.797801 18471 server_base.cc:1034] running on GCE node
I20260618 22:27:38.798374 18471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:38.799288 18471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:38.800482 18471 hybrid_clock.cc:648] HybridClock initialized: now 1781821658800427 us; error 55 us; skew 500 ppm
I20260618 22:27:38.800819 18471 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:38 dist-test-slave-0956 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821658, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:38.803898 18471 init.cc:377] Logged in from keytab as oryx/127.10.242.195@KRBTEST.COM (short username oryx)
I20260618 22:27:38.805352 18471 webserver.cc:460] Webserver started at http://127.10.242.195:34349/ using document root <none> and password file <none>
I20260618 22:27:38.806002 18471 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:38.806129 18471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:38.811328 18471 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260618 22:27:38.814082 18486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:38.815459 18471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260618 22:27:38.815640 18471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "b4a2f32ef55149f2b75701234c139912"
format_stamp: "Formatted at 2026-06-18 22:27:35 on dist-test-slave-0956"
server_key: "9f7e8151ef21f614951dfe05f2a905c6"
server_key_iv: "90089658278b2ca66a3f511adef10b57"
server_key_version: "encryptionkey@0"
I20260618 22:27:38.816629 18471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:38.838081 18471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:38.841799 18471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:38.842047 18471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:38.842986 18471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:38.844354 18471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:38.844450 18471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:38.844565 18471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:38.844633 18471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:38.876726 18471 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:46165
I20260618 22:27:38.876786 18598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:46165 every 8 connection(s)
I20260618 22:27:38.878093 18471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:27:38.886025 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 18471
Jun 18 22:27:38 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821658, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM
Jun 18 22:27:38 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821658, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.252@KRBTEST.COM
Jun 18 22:27:38 dist-test-slave-0956 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821658, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.253@KRBTEST.COM
I20260618 22:27:38.926771 18605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.881152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:34789 (local address 127.10.242.253:33505)
0618 22:27:38.906470 (+ 25318us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:38.906476 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:38.906496 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:27:38.906713 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:38.906718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:38.906789 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:38.906880 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:38.909340 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.910417 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.914568 (+ 4151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.915173 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.915291 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.920705 (+ 5414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:38.920711 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:38.920715 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:38.920758 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:38.923908 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.924691 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.924696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.924699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.924846 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.925261 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.925265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.925268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.925562 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:38.925701 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:38.926189 (+ 488us) server_negotiation.cc:299] Negotiation successful
0618 22:27:38.926392 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25189,"thread_start_us":25093,"threads_started":1}
I20260618 22:27:38.926985 18603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.880833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:49137 (local address 127.10.242.254:34635)
0618 22:27:38.901473 (+ 20640us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:38.901480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:38.905896 (+ 4416us) server_negotiation.cc:407] Connection header received
0618 22:27:38.906272 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:38.906278 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:38.906358 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:38.906456 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:38.909335 (+ 2879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.910470 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.912766 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.913318 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.913433 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.918476 (+ 5043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:38.918481 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:38.918484 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:38.918526 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:38.922873 (+ 4347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.924383 (+ 1510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.924388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.924391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.924530 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.924914 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.924919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.924922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.925315 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:38.925451 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:38.926432 (+ 981us) server_negotiation.cc:299] Negotiation successful
0618 22:27:38.926650 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20504,"thread_start_us":20406,"threads_started":1}
I20260618 22:27:38.928016 18608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.881481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:33505 (local address 127.10.242.195:34789)
0618 22:27:38.902208 (+ 20727us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:38.902274 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:38.905048 (+ 2774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:38.907059 (+ 2011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:38.907069 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:38.908232 (+ 1163us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:38.909123 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:38.909156 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.910645 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.910651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:38.914443 (+ 3792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:38.914460 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.915602 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.915608 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:38.915806 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.916800 (+ 994us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:38.916825 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:38.920528 (+ 3703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:38.924070 (+ 3542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:38.924075 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:38.924078 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:38.924565 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:38.924969 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:38.924974 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:38.924977 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:38.925147 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:38.925755 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:38.925760 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:38.925904 (+ 144us) client_negotiation.cc:769] Sending connection context
0618 22:27:38.926931 (+ 1027us) client_negotiation.cc:240] Negotiation successful
0618 22:27:38.927205 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20351,"mutex_wait_us":1978,"spinlock_wait_cycles":3968,"thread_start_us":25078,"threads_started":1}
I20260618 22:27:38.929579 18604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.881018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:43363 (local address 127.10.242.252:35943)
0618 22:27:38.896331 (+ 15313us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:38.896338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:38.907072 (+ 10734us) server_negotiation.cc:407] Connection header received
0618 22:27:38.907306 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:38.907311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:38.907389 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:38.907465 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:38.909335 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.910430 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.913074 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.913690 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.913804 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.919813 (+ 6009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:38.919818 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:38.919822 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:38.919854 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:38.923202 (+ 3348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.924007 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.924024 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.924027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.924167 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.924691 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.924696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.924699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.925043 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:38.925206 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:38.928723 (+ 3517us) server_negotiation.cc:299] Negotiation successful
0618 22:27:38.928968 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15160,"thread_start_us":15324,"threads_started":1}
I20260618 22:27:38.929966 18601 heartbeater.cc:347] Connected to a master server at 127.10.242.253:33505
I20260618 22:27:38.930099 18606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.881481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:35943 (local address 127.10.242.195:43363)
0618 22:27:38.906964 (+ 25483us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:38.906981 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:38.907163 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:38.907679 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:38.907683 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:38.908237 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:38.909115 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:38.909142 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.910646 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.910651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:38.912893 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:38.912910 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.914088 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.914094 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:38.914290 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.914891 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:38.914917 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:38.919566 (+ 4649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:38.923354 (+ 3788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:38.923359 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:38.923363 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:38.923895 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:38.924300 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:38.924304 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:38.924308 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:38.924565 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:38.925268 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:38.925277 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:38.925748 (+ 471us) client_negotiation.cc:769] Sending connection context
0618 22:27:38.926082 (+ 334us) client_negotiation.cc:240] Negotiation successful
0618 22:27:38.926603 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25351,"mutex_wait_us":665,"spinlock_wait_cycles":39296,"thread_start_us":19800,"threads_started":1}
I20260618 22:27:38.930333 18601 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:38.931003 18599 heartbeater.cc:347] Connected to a master server at 127.10.242.252:35943
I20260618 22:27:38.931144 18599 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:38.931262 18601 heartbeater.cc:511] Master 127.10.242.253:33505 requested a full tablet report, sending...
I20260618 22:27:38.931600 18599 heartbeater.cc:511] Master 127.10.242.252:35943 requested a full tablet report, sending...
I20260618 22:27:38.933041 18131 ts_manager.cc:194] Registered new tserver with Master: b4a2f32ef55149f2b75701234c139912 (127.10.242.195:46165)
I20260618 22:27:38.933640 17944 ts_manager.cc:194] Registered new tserver with Master: b4a2f32ef55149f2b75701234c139912 (127.10.242.195:46165)
I20260618 22:27:38.934382 18607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.881751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.195:49137)
0618 22:27:38.905611 (+ 23860us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:38.905632 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:38.905789 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:38.906800 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:38.906814 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:38.907938 (+ 1124us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:38.909114 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:38.909142 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.910646 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.910651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:38.912623 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:38.912634 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.913716 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.913724 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:38.914022 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.914816 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:38.914857 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:38.918246 (+ 3389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:38.923051 (+ 4805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:38.923063 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:38.923090 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:38.924242 (+ 1152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:38.924656 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:38.924661 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:38.924664 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:38.924799 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:38.925506 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:38.925511 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:38.925738 (+ 227us) client_negotiation.cc:769] Sending connection context
0618 22:27:38.926150 (+ 412us) client_negotiation.cc:240] Negotiation successful
0618 22:27:38.934102 (+ 7952us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23706,"mutex_wait_us":6013,"spinlock_wait_cycles":15616,"thread_start_us":23511,"threads_started":1}
I20260618 22:27:38.935141 18600 heartbeater.cc:347] Connected to a master server at 127.10.242.254:34635
I20260618 22:27:38.935249 18600 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:38.935628 18600 heartbeater.cc:511] Master 127.10.242.254:34635 requested a full tablet report, sending...
I20260618 22:27:38.937031 17345 ts_manager.cc:194] Re-registered known tserver with Master: b4a2f32ef55149f2b75701234c139912 (127.10.242.195:46165)
I20260618 22:27:38.938387 17345 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:49137
I20260618 22:27:38.942971 18614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.910096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60112 (local address 127.10.242.253:33505)
0618 22:27:38.919133 (+ 9037us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:38.919139 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:38.919156 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:27:38.919203 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:38.919208 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:38.919279 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:38.919580 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:38.923863 (+ 4283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.925051 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.929755 (+ 4704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.930352 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.930519 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.935179 (+ 4660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:38.935186 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:38.935192 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:38.935229 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:38.939430 (+ 4201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.940216 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.940221 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.940225 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.940718 (+ 493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.941168 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.941173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.941177 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.941517 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:38.941759 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:38.942321 (+ 562us) server_negotiation.cc:299] Negotiation successful
0618 22:27:38.942603 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7443,"thread_start_us":8824,"threads_started":1}
I20260618 22:27:38.945686 18615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.917035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34000 (local address 127.10.242.254:34635)
0618 22:27:38.919812 (+ 2777us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:38.919818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:38.927938 (+ 8120us) server_negotiation.cc:407] Connection header received
0618 22:27:38.928137 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:38.928142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:38.928214 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:38.928287 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:38.932431 (+ 4144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.933542 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.935318 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.935945 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.936074 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.939361 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:38.939367 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:38.939371 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:38.939406 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:38.942707 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.943694 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.943700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.943705 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.943835 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.944302 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.944305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.944308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.944560 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:38.944799 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:38.945133 (+ 334us) server_negotiation.cc:299] Negotiation successful
0618 22:27:38.945425 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2612,"thread_start_us":2958,"threads_started":1}
I20260618 22:27:38.949015 18616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.920525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41392 (local address 127.10.242.252:35943)
0618 22:27:38.921189 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:38.921194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:38.928181 (+ 6987us) server_negotiation.cc:407] Connection header received
0618 22:27:38.928390 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:38.928395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:38.928463 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:38.928531 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:38.936094 (+ 7563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.937484 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.939382 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.940067 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.940152 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.943359 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:38.943366 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:38.943371 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:38.943410 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:38.946402 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.947157 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.947162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.947165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.947312 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:38.947742 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:38.947748 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:38.947751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:38.948037 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:38.948250 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:38.948549 (+ 299us) server_negotiation.cc:299] Negotiation successful
0618 22:27:38.948734 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":312,"threads_started":1}
I20260618 22:27:38.949926 17345 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:27:38.956918 17345 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:27:38.984901 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.974621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:46165 (local address 127.0.0.1:34116)
0618 22:27:38.975708 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:38.975728 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:38.975861 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:38.977474 (+ 1613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:38.977479 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:38.977505 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:38.977837 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:38.977845 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.980397 (+ 2552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.980403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:38.982874 (+ 2471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:38.982885 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.984127 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.984133 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:38.984302 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.984333 (+ 31us) client_negotiation.cc:769] Sending connection context
0618 22:27:38.984418 (+ 85us) client_negotiation.cc:240] Negotiation successful
0618 22:27:38.984552 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":964,"thread_start_us":833,"threads_started":1}
I20260618 22:27:38.985998 18629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.975781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:42531 (local address 127.0.0.1:49694)
0618 22:27:38.976720 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:38.976740 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:38.976840 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:38.978424 (+ 1584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:38.978428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:38.978449 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:38.978721 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:38.978729 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.979874 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.979879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:38.982721 (+ 2842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:38.982736 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.984580 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.984586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:38.985151 (+ 565us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.985191 (+ 40us) client_negotiation.cc:769] Sending connection context
0618 22:27:38.985258 (+ 67us) client_negotiation.cc:240] Negotiation successful
0618 22:27:38.985378 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"mutex_wait_us":171,"thread_start_us":701,"threads_started":1}
I20260618 22:27:38.986104 18630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.975994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49694 (local address 127.10.242.194:42531)
0618 22:27:38.976423 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:38.976432 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:38.977972 (+ 1540us) server_negotiation.cc:407] Connection header received
0618 22:27:38.978049 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:38.978055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:38.978261 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:38.978421 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:38.978851 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.979755 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.982896 (+ 3141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.984374 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.984561 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.984965 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:38.985553 (+ 588us) server_negotiation.cc:299] Negotiation successful
0618 22:27:38.985793 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":203,"threads_started":1}
I20260618 22:27:38.987526 18628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.974856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34116 (local address 127.10.242.195:46165)
0618 22:27:38.975513 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:38.975524 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:38.975901 (+ 377us) server_negotiation.cc:407] Connection header received
0618 22:27:38.976022 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:38.976031 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:38.976328 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:38.976749 (+ 421us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:38.978040 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.979905 (+ 1865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.983008 (+ 3103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.983968 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.984120 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.984321 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:38.986140 (+ 1819us) server_negotiation.cc:299] Negotiation successful
0618 22:27:38.987233 (+ 1093us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":283,"threads_started":1}
I20260618 22:27:38.989946 18393 tablet_service.cc:1467] Processing CreateTablet for tablet c4ed3f7df65e46678c03dc4da27b7db7 (DEFAULT_TABLE table=test-table [id=e0fa9f440ebb4f20b8f619d52108bc7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:38.992097 18393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4ed3f7df65e46678c03dc4da27b7db7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:38.992857 18534 tablet_service.cc:1467] Processing CreateTablet for tablet c4ed3f7df65e46678c03dc4da27b7db7 (DEFAULT_TABLE table=test-table [id=e0fa9f440ebb4f20b8f619d52108bc7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:38.994405 18534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4ed3f7df65e46678c03dc4da27b7db7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:39.000479 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.976798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:46667 (local address 127.0.0.1:39434)
0618 22:27:38.977463 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:38.977478 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:38.977570 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:38.993353 (+ 15783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:38.993358 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:38.993386 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:38.994013 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:38.994028 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.995807 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.995813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:38.997812 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:38.997827 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.999680 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.999686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:38.999852 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.999888 (+ 36us) client_negotiation.cc:769] Sending connection context
0618 22:27:38.999987 (+ 99us) client_negotiation.cc:240] Negotiation successful
0618 22:27:39.000129 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":320,"threads_started":1}
I20260618 22:27:39.000496 18632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:38.977243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39434 (local address 127.10.242.193:46667)
0618 22:27:38.992693 (+ 15450us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:38.992702 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:38.992731 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:27:38.992825 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:38.992832 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:38.993131 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:38.993319 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:38.994223 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.995661 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.997980 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:38.999466 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:38.999649 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:38.999810 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:39.000037 (+ 227us) server_negotiation.cc:299] Negotiation successful
0618 22:27:39.000199 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15039,"thread_start_us":15237,"threads_started":1}
I20260618 22:27:39.004714 18243 tablet_service.cc:1467] Processing CreateTablet for tablet c4ed3f7df65e46678c03dc4da27b7db7 (DEFAULT_TABLE table=test-table [id=e0fa9f440ebb4f20b8f619d52108bc7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:39.006505 18243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4ed3f7df65e46678c03dc4da27b7db7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:39.007331 18633 tablet_bootstrap.cc:492] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144: Bootstrap starting.
I20260618 22:27:39.009313 18634 tablet_bootstrap.cc:492] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912: Bootstrap starting.
I20260618 22:27:39.010982 18633 tablet_bootstrap.cc:654] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:39.012183 18633 log.cc:824] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:39.012575 18634 tablet_bootstrap.cc:654] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:39.013957 18634 log.cc:824] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:39.016312 18633 tablet_bootstrap.cc:492] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144: No bootstrap required, opened a new log
I20260618 22:27:39.016597 18633 ts_tablet_manager.cc:1397] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260618 22:27:39.019575 18634 tablet_bootstrap.cc:492] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912: No bootstrap required, opened a new log
I20260618 22:27:39.019833 18634 ts_tablet_manager.cc:1397] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260618 22:27:39.020094 18637 tablet_bootstrap.cc:492] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c: Bootstrap starting.
I20260618 22:27:39.023392 18637 tablet_bootstrap.cc:654] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:39.024353 18637 log.cc:824] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:39.026527 18633 raft_consensus.cc:348] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } } peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } }
I20260618 22:27:39.026985 18633 raft_consensus.cc:374] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:39.027127 18633 raft_consensus.cc:729] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb54f92e5ac4441bed6f5bb6ea32144, State: Initialized, Role: FOLLOWER
I20260618 22:27:39.027490 18637 tablet_bootstrap.cc:492] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c: No bootstrap required, opened a new log
I20260618 22:27:39.027755 18637 ts_tablet_manager.cc:1397] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260618 22:27:39.027850 18633 consensus_queue.cc:260] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } } peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } }
I20260618 22:27:39.028891 18459 heartbeater.cc:503] Master 127.10.242.254:34635 was elected leader, sending a full tablet report...
I20260618 22:27:39.032723 18634 raft_consensus.cc:348] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } } peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } }
I20260618 22:27:39.033195 18634 raft_consensus.cc:374] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:39.033306 18634 raft_consensus.cc:729] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4a2f32ef55149f2b75701234c139912, State: Initialized, Role: FOLLOWER
I20260618 22:27:39.033929 18634 consensus_queue.cc:260] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } } peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } }
I20260618 22:27:39.034494 18633 ts_tablet_manager.cc:1428] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144: Time spent starting tablet: real 0.018s user 0.014s sys 0.000s
I20260618 22:27:39.035410 18600 heartbeater.cc:503] Master 127.10.242.254:34635 was elected leader, sending a full tablet report...
I20260618 22:27:39.038264 18634 ts_tablet_manager.cc:1428] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912: Time spent starting tablet: real 0.018s user 0.007s sys 0.008s
I20260618 22:27:39.041968 18637 raft_consensus.cc:348] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } } peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } }
I20260618 22:27:39.042308 18637 raft_consensus.cc:374] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:39.042378 18637 raft_consensus.cc:729] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5899712e99ff4d88bb415b836db9967c, State: Initialized, Role: FOLLOWER
I20260618 22:27:39.043267 18637 consensus_queue.cc:260] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } } peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } }
I20260618 22:27:39.044984 18637 ts_tablet_manager.cc:1428] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c: Time spent starting tablet: real 0.017s user 0.012s sys 0.000s
I20260618 22:27:39.061617 18641 raft_consensus.cc:482] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:27:39.061890 18641 raft_consensus.cc:504] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } } peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } }
I20260618 22:27:39.063457 18641 leader_election.cc:288] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4a2f32ef55149f2b75701234c139912 (127.10.242.195:46165), 0fb54f92e5ac4441bed6f5bb6ea32144 (127.10.242.194:42531)
I20260618 22:27:39.070302 18628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:39.063827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:44205 (local address 127.10.242.195:46165)
0618 22:27:39.064032 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:39.064039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:39.064510 (+ 471us) server_negotiation.cc:407] Connection header received
0618 22:27:39.064704 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:39.064710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:39.064783 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:39.064869 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:39.065451 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:39.066118 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:39.068386 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:39.069319 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:39.069464 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:39.069538 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:39.070026 (+ 488us) server_negotiation.cc:299] Negotiation successful
0618 22:27:39.070123 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260618 22:27:39.070525 18642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:39.063835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:46165 (local address 127.10.242.193:44205)
0618 22:27:39.064395 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:39.064410 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:39.064569 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:39.064911 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:39.064917 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:39.064955 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:39.065324 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:39.065334 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:39.066274 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:39.066279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:39.068222 (+ 1943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:39.068238 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:39.069670 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:39.069677 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:39.069842 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:39.069877 (+ 35us) client_negotiation.cc:769] Sending connection context
0618 22:27:39.069970 (+ 93us) client_negotiation.cc:240] Negotiation successful
0618 22:27:39.070096 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":313,"threads_started":1}
I20260618 22:27:39.071578 18554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ed3f7df65e46678c03dc4da27b7db7" candidate_uuid: "5899712e99ff4d88bb415b836db9967c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a2f32ef55149f2b75701234c139912" is_pre_election: true
I20260618 22:27:39.071822 18630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:39.064706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:36367 (local address 127.10.242.194:42531)
0618 22:27:39.064897 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:39.064902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:39.065353 (+ 451us) server_negotiation.cc:407] Connection header received
0618 22:27:39.065512 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:39.065517 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:39.065572 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:39.065641 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:39.066092 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:39.067032 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:39.069379 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:39.071057 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:39.071166 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:39.071214 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:39.071546 (+ 332us) server_negotiation.cc:299] Negotiation successful
0618 22:27:39.071634 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:27:39.072093 18643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:39.064510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:42531 (local address 127.10.242.193:36367)
0618 22:27:39.065265 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:39.065278 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:39.065392 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:39.065679 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:39.065683 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:39.065706 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:39.065954 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:39.065962 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:39.067206 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:39.067211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:39.069191 (+ 1980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:39.069218 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:39.071230 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:39.071236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:39.071402 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:39.071436 (+ 34us) client_negotiation.cc:769] Sending connection context
0618 22:27:39.071531 (+ 95us) client_negotiation.cc:240] Negotiation successful
0618 22:27:39.071653 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":1,"thread_start_us":486,"threads_started":1}
I20260618 22:27:39.072154 18554 raft_consensus.cc:2454] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5899712e99ff4d88bb415b836db9967c in term 0.
I20260618 22:27:39.072784 18197 leader_election.cc:302] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5899712e99ff4d88bb415b836db9967c, b4a2f32ef55149f2b75701234c139912; no voters:
I20260618 22:27:39.072923 18413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ed3f7df65e46678c03dc4da27b7db7" candidate_uuid: "5899712e99ff4d88bb415b836db9967c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" is_pre_election: true
I20260618 22:27:39.073179 18641 raft_consensus.cc:2781] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:27:39.073436 18413 raft_consensus.cc:2454] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5899712e99ff4d88bb415b836db9967c in term 0.
I20260618 22:27:39.073581 18641 raft_consensus.cc:482] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:27:39.073705 18641 raft_consensus.cc:3037] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:39.075938 18641 raft_consensus.cc:504] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } } peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } }
I20260618 22:27:39.076442 18641 leader_election.cc:288] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [CANDIDATE]: Term 1 election: Requested vote from peers b4a2f32ef55149f2b75701234c139912 (127.10.242.195:46165), 0fb54f92e5ac4441bed6f5bb6ea32144 (127.10.242.194:42531)
I20260618 22:27:39.076869 18554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ed3f7df65e46678c03dc4da27b7db7" candidate_uuid: "5899712e99ff4d88bb415b836db9967c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a2f32ef55149f2b75701234c139912"
I20260618 22:27:39.076895 18413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ed3f7df65e46678c03dc4da27b7db7" candidate_uuid: "5899712e99ff4d88bb415b836db9967c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144"
I20260618 22:27:39.077040 18554 raft_consensus.cc:3037] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:39.077050 18413 raft_consensus.cc:3037] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:39.079159 18554 raft_consensus.cc:2454] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5899712e99ff4d88bb415b836db9967c in term 1.
I20260618 22:27:39.079166 18413 raft_consensus.cc:2454] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5899712e99ff4d88bb415b836db9967c in term 1.
I20260618 22:27:39.079466 18197 leader_election.cc:302] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5899712e99ff4d88bb415b836db9967c, b4a2f32ef55149f2b75701234c139912; no voters:
I20260618 22:27:39.079721 18641 raft_consensus.cc:2781] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:39.080117 18641 raft_consensus.cc:686] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [term 1 LEADER]: Becoming Leader. State: Replica: 5899712e99ff4d88bb415b836db9967c, State: Running, Role: LEADER
I20260618 22:27:39.080654 18641 consensus_queue.cc:237] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } } peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } }
I20260618 22:27:39.086163 17344 catalog_manager.cc:5495] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5899712e99ff4d88bb415b836db9967c (127.10.242.193). New cstate: current_term: 1 leader_uuid: "5899712e99ff4d88bb415b836db9967c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260618 22:27:39.110612 18632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:39.105163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39444 (local address 127.10.242.193:46667)
0618 22:27:39.105360 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:39.105366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:39.105440 (+ 74us) server_negotiation.cc:407] Connection header received
0618 22:27:39.105673 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:39.105677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:39.105753 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:39.105877 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:27:39.106242 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:39.106900 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:39.108162 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:39.108543 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:39.108644 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:39.109152 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:27:39.109676 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:27:39.109811 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:39.109886 (+ 75us) server_negotiation.cc:299] Negotiation successful
0618 22:27:39.109962 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260618 22:27:39.119125 18413 raft_consensus.cc:1261] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144 [term 1 FOLLOWER]: Refusing update from remote peer 5899712e99ff4d88bb415b836db9967c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:39.119124 18554 raft_consensus.cc:1261] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912 [term 1 FOLLOWER]: Refusing update from remote peer 5899712e99ff4d88bb415b836db9967c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:39.120187 18644 consensus_queue.cc:1035] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:39.120532 18641 consensus_queue.cc:1035] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 22:27:39.130355 18602 tablet.cc:2369] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:27:39.140216 18647 mvcc.cc:205] Tried to move back new op lower bound from 7298341515741011968 to 7298341515599814656. Current Snapshot: MvccSnapshot[applied={T|T < 7298341515741011968}]
I20260618 22:27:39.152187 17345 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:39.152525 17345 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:39.158309 17345 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7a255bfe590d478991390a7b3b7e9b80: Sending DeleteTablet for 3 replicas of tablet c4ed3f7df65e46678c03dc4da27b7db7
I20260618 22:27:39.159546 18534 tablet_service.cc:1514] Processing DeleteTablet for tablet c4ed3f7df65e46678c03dc4da27b7db7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:39 UTC) from {username='oryx', principal='oryx/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:34116
I20260618 22:27:39.159683 18393 tablet_service.cc:1514] Processing DeleteTablet for tablet c4ed3f7df65e46678c03dc4da27b7db7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:39 UTC) from {username='oryx', principal='oryx/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:49694
I20260618 22:27:39.160753 18243 tablet_service.cc:1514] Processing DeleteTablet for tablet c4ed3f7df65e46678c03dc4da27b7db7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:39 UTC) from {username='oryx', principal='oryx/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:39434
I20260618 22:27:39.170663 18657 tablet_replica.cc:330] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912: stopping tablet replica
I20260618 22:27:39.171175 18657 raft_consensus.cc:2229] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:39.171697 18657 raft_consensus.cc:2258] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:39.172495 18658 tablet_replica.cc:330] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144: stopping tablet replica
I20260618 22:27:39.172972 18658 raft_consensus.cc:2229] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:39.173463 18658 raft_consensus.cc:2258] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:39.174309 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 17313
I20260618 22:27:39.175484 17375 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:39.176192 18658 ts_tablet_manager.cc:1905] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:39.181365 18657 ts_tablet_manager.cc:1905] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:39.182089 18659 tablet_replica.cc:330] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c: stopping tablet replica
I20260618 22:27:39.182536 18659 raft_consensus.cc:2229] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:27:39.183251 18659 raft_consensus.cc:2258] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:39.188872 18659 ts_tablet_manager.cc:1905] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:39.192335 18614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:39.163346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60114 (local address 127.10.242.253:33505)
0618 22:27:39.163544 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:39.163551 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:39.172590 (+ 9039us) server_negotiation.cc:407] Connection header received
0618 22:27:39.175797 (+ 3207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:39.175803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:39.175887 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:39.175993 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:39.178094 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:39.179129 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:39.180993 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:39.181644 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:39.181768 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:39.185121 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:39.185128 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:39.185132 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:39.185174 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:39.188534 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:39.189941 (+ 1407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:39.189947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:39.189950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:39.190103 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:39.190544 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:39.190549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:39.190551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:39.190912 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:39.191125 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:39.191724 (+ 599us) server_negotiation.cc:299] Negotiation successful
0618 22:27:39.191946 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:27:39.192744 18658 ts_tablet_manager.cc:1918] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:27:39.192922 18658 log.cc:1196] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/c4ed3f7df65e46678c03dc4da27b7db7
I20260618 22:27:39.193621 18658 ts_tablet_manager.cc:1939] T c4ed3f7df65e46678c03dc4da27b7db7 P 0fb54f92e5ac4441bed6f5bb6ea32144: Deleting consensus metadata
I20260618 22:27:39.194279 18657 ts_tablet_manager.cc:1918] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:27:39.194460 18657 log.cc:1196] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/c4ed3f7df65e46678c03dc4da27b7db7
I20260618 22:27:39.195014 18657 ts_tablet_manager.cc:1939] T c4ed3f7df65e46678c03dc4da27b7db7 P b4a2f32ef55149f2b75701234c139912: Deleting consensus metadata
I20260618 22:27:39.201440 18659 ts_tablet_manager.cc:1918] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:27:39.201646 18659 log.cc:1196] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/c4ed3f7df65e46678c03dc4da27b7db7
I20260618 22:27:39.202188 18659 ts_tablet_manager.cc:1939] T c4ed3f7df65e46678c03dc4da27b7db7 P 5899712e99ff4d88bb415b836db9967c: Deleting consensus metadata
I20260618 22:27:39.212847 18616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:39.173291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41406 (local address 127.10.242.252:35943)
0618 22:27:39.173538 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:39.173546 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:39.193574 (+ 20028us) server_negotiation.cc:407] Connection header received
0618 22:27:39.194325 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:39.194332 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:39.194435 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:39.194525 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:39.201278 (+ 6753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:39.202340 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:39.204153 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:39.204557 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:39.204673 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:39.207661 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:39.207665 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:39.207668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:39.207702 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:39.210262 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:39.210903 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:39.210908 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:39.210910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:39.211039 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:39.211387 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:39.211391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:39.211393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:39.211741 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:39.211949 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:39.212329 (+ 380us) server_negotiation.cc:299] Negotiation successful
0618 22:27:39.212546 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:27:39.341859 17330 catalog_manager.cc:4841] TS 5899712e99ff4d88bb415b836db9967c (127.10.242.193:46667): tablet c4ed3f7df65e46678c03dc4da27b7db7 (table test-table [id=e0fa9f440ebb4f20b8f619d52108bc7b]) successfully deleted
I20260618 22:27:39.342216 17330 catalog_manager.cc:4841] TS 0fb54f92e5ac4441bed6f5bb6ea32144 (127.10.242.194:42531): tablet c4ed3f7df65e46678c03dc4da27b7db7 (table test-table [id=e0fa9f440ebb4f20b8f619d52108bc7b]) successfully deleted
I20260618 22:27:39.342429 17330 catalog_manager.cc:4841] TS b4a2f32ef55149f2b75701234c139912 (127.10.242.195:46165): tablet c4ed3f7df65e46678c03dc4da27b7db7 (table test-table [id=e0fa9f440ebb4f20b8f619d52108bc7b]) successfully deleted
I20260618 22:27:39.342918 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 17313
I20260618 22:27:39.384459 18614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:39.364579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60124 (local address 127.10.242.253:33505)
0618 22:27:39.364745 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:39.364752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:39.368561 (+ 3809us) server_negotiation.cc:407] Connection header received
0618 22:27:39.369073 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:39.369175 (+ 102us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:39.369299 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:39.369379 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:39.372591 (+ 3212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:39.373521 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:39.374823 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:39.375245 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:39.375353 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:39.378205 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:39.378212 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:39.378218 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:39.378259 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:39.381024 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:39.381815 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:39.381820 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:39.381824 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:39.381980 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:39.382379 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:39.382384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:39.382387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:39.382702 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:39.382925 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:39.383888 (+ 963us) server_negotiation.cc:299] Negotiation successful
0618 22:27:39.384113 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:27:39.387831 18616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:39.364343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41410 (local address 127.10.242.252:35943)
0618 22:27:39.364748 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:39.364754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:39.372485 (+ 7731us) server_negotiation.cc:407] Connection header received
0618 22:27:39.373184 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:39.373191 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:39.373286 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:39.373381 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:39.375514 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:39.376606 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:39.378377 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:39.378803 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:39.378910 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:39.381999 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:39.382005 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:39.382009 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:39.382048 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:39.385277 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:39.385944 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:39.385949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:39.385953 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:39.386065 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:39.386444 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:39.386449 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:39.386453 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:39.386766 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:39.386966 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:39.387276 (+ 310us) server_negotiation.cc:299] Negotiation successful
0618 22:27:39.387536 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274}
I20260618 22:27:40.139390 18669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.138539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.195:34081)
0618 22:27:40.138976 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.139124 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":232,"threads_started":1}
W20260618 22:27:40.139801 18600 heartbeater.cc:644] Failed to heartbeat to 127.10.242.254:34635 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
I20260618 22:27:40.141647 18670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.140872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.194:56769)
0618 22:27:40.141337 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.141466 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":257,"threads_started":1}
W20260618 22:27:40.141999 18459 heartbeater.cc:644] Failed to heartbeat to 127.10.242.254:34635 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
I20260618 22:27:40.146282 18671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.145583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.193:32907)
0618 22:27:40.145995 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.146096 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":222,"threads_started":1}
W20260618 22:27:40.146680 18309 heartbeater.cc:644] Failed to heartbeat to 127.10.242.254:34635 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
I20260618 22:27:40.660303 18672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a255bfe590d478991390a7b3b7e9b80)
I20260618 22:27:40.660392 18673 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a255bfe590d478991390a7b3b7e9b80)
I20260618 22:27:40.660606 18672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false } }
I20260618 22:27:40.660604 18673 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false } }
I20260618 22:27:40.662142 18672 leader_election.cc:288] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a255bfe590d478991390a7b3b7e9b80 (127.10.242.254:34635), bfd01e9042ff4ee3819c16bcee8dd973 (127.10.242.252:35943)
I20260618 22:27:40.662864 18673 leader_election.cc:288] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a255bfe590d478991390a7b3b7e9b80 (127.10.242.254:34635), 468d5c19d3c54b7bbd0d7122ea722111 (127.10.242.253:33505)
I20260618 22:27:40.663240 18674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.662245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.0.0.1:34436)
0618 22:27:40.662834 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.662976 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":388,"threads_started":1}
W20260618 22:27:40.663551 17931 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
I20260618 22:27:40.669960 18675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.662227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.0.0.1:34434)
0618 22:27:40.663219 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.669617 (+ 6398us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":886,"thread_start_us":671,"threads_started":1}
W20260618 22:27:40.670970 18118 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
I20260618 22:27:40.671934 18674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.662571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:35943 (local address 127.0.0.1:41424)
0618 22:27:40.663440 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.663458 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:40.663603 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:40.664458 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:40.664464 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:40.664492 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:40.664761 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:40.664772 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.666500 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.666506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:40.668798 (+ 2292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:40.668816 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.671169 (+ 2353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.671176 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:40.671366 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:40.671411 (+ 45us) client_negotiation.cc:769] Sending connection context
0618 22:27:40.671485 (+ 74us) client_negotiation.cc:240] Negotiation successful
0618 22:27:40.671639 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":8324,"threads_started":1}
I20260618 22:27:40.672430 18678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.663118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41424 (local address 127.10.242.252:35943)
0618 22:27:40.664109 (+ 991us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:40.664115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:40.664133 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:27:40.664196 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:40.664201 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:40.664287 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:40.664890 (+ 603us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:40.665071 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.666325 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.669012 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.670409 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.671803 (+ 1394us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:40.671871 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:40.671920 (+ 49us) server_negotiation.cc:299] Negotiation successful
0618 22:27:40.672056 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":874,"thread_start_us":741,"threads_started":1}
I20260618 22:27:40.673319 18675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.672917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.0.0.1:34442)
0618 22:27:40.673060 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.673176 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260618 22:27:40.673424 18141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "468d5c19d3c54b7bbd0d7122ea722111" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" is_pre_election: true
W20260618 22:27:40.673558 18118 leader_election.cc:334] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7a255bfe590d478991390a7b3b7e9b80 (127.10.242.254:34635): Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
I20260618 22:27:40.673828 18141 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468d5c19d3c54b7bbd0d7122ea722111 in term 1.
I20260618 22:27:40.674238 18674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.673807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.0.0.1:34450)
0618 22:27:40.673985 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.674072 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260618 22:27:40.674454 17931 leader_election.cc:334] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7a255bfe590d478991390a7b3b7e9b80 (127.10.242.254:34635): Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
I20260618 22:27:40.674674 17931 leader_election.cc:302] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468d5c19d3c54b7bbd0d7122ea722111, bfd01e9042ff4ee3819c16bcee8dd973; no voters: 7a255bfe590d478991390a7b3b7e9b80
I20260618 22:27:40.675027 18672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260618 22:27:40.675155 18672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a255bfe590d478991390a7b3b7e9b80)
I20260618 22:27:40.675256 18672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:27:40.677707 18672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false } }
I20260618 22:27:40.678366 18672 leader_election.cc:288] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [CANDIDATE]: Term 2 election: Requested vote from peers 7a255bfe590d478991390a7b3b7e9b80 (127.10.242.254:34635), bfd01e9042ff4ee3819c16bcee8dd973 (127.10.242.252:35943)
I20260618 22:27:40.678844 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.662693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:33505 (local address 127.0.0.1:60134)
0618 22:27:40.670841 (+ 8148us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.670861 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:40.670970 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:40.673449 (+ 2479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:40.673454 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:40.673484 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:40.673783 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:40.673794 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.675247 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.675252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:40.676522 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:40.676535 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.678096 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.678101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:40.678242 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:40.678274 (+ 32us) client_negotiation.cc:769] Sending connection context
0618 22:27:40.678377 (+ 103us) client_negotiation.cc:240] Negotiation successful
0618 22:27:40.678510 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8036,"thread_start_us":7924,"threads_started":1}
I20260618 22:27:40.678802 18141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "468d5c19d3c54b7bbd0d7122ea722111" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bfd01e9042ff4ee3819c16bcee8dd973"
I20260618 22:27:40.679028 18674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.678630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.0.0.1:34462)
0618 22:27:40.678765 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.678872 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260618 22:27:40.679036 18141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:27:40.679239 18681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.671183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60134 (local address 127.10.242.253:33505)
0618 22:27:40.672939 (+ 1756us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:40.672944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:40.672965 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:27:40.673022 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:40.673027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:40.673129 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:40.673231 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:40.674016 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.675068 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.676678 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.677880 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.678070 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:40.678137 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:40.678766 (+ 629us) server_negotiation.cc:299] Negotiation successful
0618 22:27:40.678896 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1653,"thread_start_us":1462,"threads_started":1}
I20260618 22:27:40.679785 17953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "468d5c19d3c54b7bbd0d7122ea722111" is_pre_election: true
I20260618 22:27:40.680131 17953 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bfd01e9042ff4ee3819c16bcee8dd973 in current term 2: Already voted for candidate 468d5c19d3c54b7bbd0d7122ea722111 in this term.
I20260618 22:27:40.680291 18674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.679872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.0.0.1:34472)
0618 22:27:40.680031 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.680112 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260618 22:27:40.680457 17931 leader_election.cc:334] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7a255bfe590d478991390a7b3b7e9b80 (127.10.242.254:34635): Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
I20260618 22:27:40.680801 18117 leader_election.cc:302] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfd01e9042ff4ee3819c16bcee8dd973; no voters: 468d5c19d3c54b7bbd0d7122ea722111, 7a255bfe590d478991390a7b3b7e9b80
I20260618 22:27:40.681654 18141 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468d5c19d3c54b7bbd0d7122ea722111 in term 2.
I20260618 22:27:40.681825 18673 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 22:27:40.682092 17928 leader_election.cc:302] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468d5c19d3c54b7bbd0d7122ea722111, bfd01e9042ff4ee3819c16bcee8dd973; no voters: 7a255bfe590d478991390a7b3b7e9b80
I20260618 22:27:40.682370 18672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 2 FOLLOWER]: Leader election won for term 2
I20260618 22:27:40.682814 18672 raft_consensus.cc:686] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [term 2 LEADER]: Becoming Leader. State: Replica: 468d5c19d3c54b7bbd0d7122ea722111, State: Running, Role: LEADER
I20260618 22:27:40.683161 18672 consensus_queue.cc:237] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false } }
I20260618 22:27:40.685137 18682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 468d5c19d3c54b7bbd0d7122ea722111. Latest consensus state: current_term: 2 leader_uuid: "468d5c19d3c54b7bbd0d7122ea722111" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false } } }
I20260618 22:27:40.685448 18682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:40.686080 18684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:27:40.689673 18684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:27:40.690441 18684 catalog_manager.cc:1259] Loaded cluster ID: c4e1d5b328594abba209eac27d127622
I20260618 22:27:40.690532 18684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:27:40.691610 18684 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:27:40.692401 18684 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: Loaded TSK: 0
I20260618 22:27:40.693266 18684 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:27:40.720959 17944 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:27:40.727763 17944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:27:40.732143 18674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.731753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.0.0.1:34488)
0618 22:27:40.731925 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.732010 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260618 22:27:40.732254 18141 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [term 2 FOLLOWER]: Refusing update from remote peer 468d5c19d3c54b7bbd0d7122ea722111: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260618 22:27:40.733027 18682 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260618 22:27:40.733321 18674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.732851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.0.0.1:34492)
0618 22:27:40.733049 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.733163 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260618 22:27:40.733496 17931 consensus_peers.cc:487] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 -> Peer 7a255bfe590d478991390a7b3b7e9b80 (127.10.242.254:34635): Couldn't send request to peer 7a255bfe590d478991390a7b3b7e9b80. Status: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260618 22:27:40.735895 18673 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 468d5c19d3c54b7bbd0d7122ea722111. Latest consensus state: current_term: 2 leader_uuid: "468d5c19d3c54b7bbd0d7122ea722111" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false } } }
I20260618 22:27:40.736172 18673 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:27:40.737427 18682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "468d5c19d3c54b7bbd0d7122ea722111" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false } } }
I20260618 22:27:40.737771 18682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:40.738437 18686 mvcc.cc:205] Tried to move back new op lower bound from 7298341522352427008 to 7298341522161315840. Current Snapshot: MvccSnapshot[applied={T|T < 7298341522352427008}]
I20260618 22:27:40.738336 18673 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "468d5c19d3c54b7bbd0d7122ea722111" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7a255bfe590d478991390a7b3b7e9b80" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 34635 } } peers { permanent_uuid: "468d5c19d3c54b7bbd0d7122ea722111" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 33505 } attrs { promote: false } } peers { permanent_uuid: "bfd01e9042ff4ee3819c16bcee8dd973" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 35943 } attrs { promote: false } } }
I20260618 22:27:40.740921 18673 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd01e9042ff4ee3819c16bcee8dd973 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:27:40.756491 18674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.745381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:46667 (local address 127.0.0.1:39448)
0618 22:27:40.745642 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.745666 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:40.745773 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:40.746218 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:40.746223 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:40.746247 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:40.746451 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:40.746463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.752752 (+ 6289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.752759 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:40.754452 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:40.754464 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.755785 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.755790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:40.755957 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:40.755987 (+ 30us) client_negotiation.cc:769] Sending connection context
0618 22:27:40.756063 (+ 76us) client_negotiation.cc:240] Negotiation successful
0618 22:27:40.756191 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260618 22:27:40.756778 18691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.745446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39448 (local address 127.10.242.193:46667)
0618 22:27:40.745874 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:40.745880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:40.745899 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:27:40.745994 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:40.745999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:40.746073 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:40.746583 (+ 510us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:40.746769 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.747877 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.754621 (+ 6744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.755631 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.755773 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:40.755847 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:40.756297 (+ 450us) server_negotiation.cc:299] Negotiation successful
0618 22:27:40.756430 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":222,"threads_started":1}
I20260618 22:27:40.757923 18243 tablet_service.cc:1467] Processing CreateTablet for tablet a9bed8fbdaca43e3832444f7f68e23a1 (DEFAULT_TABLE table=test-table [id=b4e3af3c3e3649b5aefac4fa6a70be16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:40.758410 18243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9bed8fbdaca43e3832444f7f68e23a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:40.759133 18676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.746188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:46165 (local address 127.0.0.1:34124)
0618 22:27:40.746350 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.746368 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:40.746475 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:40.749910 (+ 3435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:40.749917 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:40.749949 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:40.753205 (+ 3256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:40.753219 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.754787 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.754792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:40.756691 (+ 1899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:40.756705 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.758383 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.758390 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:40.758585 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:40.758621 (+ 36us) client_negotiation.cc:769] Sending connection context
0618 22:27:40.758700 (+ 79us) client_negotiation.cc:240] Negotiation successful
0618 22:27:40.758824 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":4}
I20260618 22:27:40.759953 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.746296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34124 (local address 127.10.242.195:46165)
0618 22:27:40.749499 (+ 3203us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:40.749506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:40.749529 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:27:40.749601 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:40.749607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:40.749708 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:40.749858 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:40.753435 (+ 3577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.754623 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.757039 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.758144 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.759385 (+ 1241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:40.759457 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:40.759507 (+ 50us) server_negotiation.cc:299] Negotiation successful
0618 22:27:40.759639 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3071,"thread_start_us":2946,"threads_started":1}
I20260618 22:27:40.760581 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.746451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:42531 (local address 127.0.0.1:49696)
0618 22:27:40.753044 (+ 6593us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.753067 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:40.753209 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:40.755194 (+ 1985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:40.755199 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:40.755230 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:40.755604 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:40.755616 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.756888 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.756893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:40.758630 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:40.758641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.759821 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.759827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:40.759985 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:40.760025 (+ 40us) client_negotiation.cc:769] Sending connection context
0618 22:27:40.760105 (+ 80us) client_negotiation.cc:240] Negotiation successful
0618 22:27:40.760239 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6445,"mutex_wait_us":76,"thread_start_us":6284,"threads_started":1}
I20260618 22:27:40.760586 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.746800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49696 (local address 127.10.242.194:42531)
0618 22:27:40.754909 (+ 8109us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:40.754915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:40.754933 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:27:40.754994 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:40.754999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:40.755067 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:40.755159 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:40.755784 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.756764 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.758778 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.759633 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.759780 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:40.759850 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:40.760170 (+ 320us) server_negotiation.cc:299] Negotiation successful
0618 22:27:40.760310 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7992,"thread_start_us":7904,"threads_started":1}
I20260618 22:27:40.761171 18534 tablet_service.cc:1467] Processing CreateTablet for tablet a9bed8fbdaca43e3832444f7f68e23a1 (DEFAULT_TABLE table=test-table [id=b4e3af3c3e3649b5aefac4fa6a70be16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:40.761620 18534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9bed8fbdaca43e3832444f7f68e23a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:40.761868 18393 tablet_service.cc:1467] Processing CreateTablet for tablet a9bed8fbdaca43e3832444f7f68e23a1 (DEFAULT_TABLE table=test-table [id=b4e3af3c3e3649b5aefac4fa6a70be16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:40.762252 18393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9bed8fbdaca43e3832444f7f68e23a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:40.765383 18695 tablet_bootstrap.cc:492] T a9bed8fbdaca43e3832444f7f68e23a1 P 5899712e99ff4d88bb415b836db9967c: Bootstrap starting.
I20260618 22:27:40.767210 18695 tablet_bootstrap.cc:654] T a9bed8fbdaca43e3832444f7f68e23a1 P 5899712e99ff4d88bb415b836db9967c: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:40.770603 18696 tablet_bootstrap.cc:492] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912: Bootstrap starting.
I20260618 22:27:40.771127 18697 tablet_bootstrap.cc:492] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144: Bootstrap starting.
I20260618 22:27:40.772344 18696 tablet_bootstrap.cc:654] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:40.772719 18697 tablet_bootstrap.cc:654] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:40.774570 18696 tablet_bootstrap.cc:492] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912: No bootstrap required, opened a new log
I20260618 22:27:40.774658 18695 tablet_bootstrap.cc:492] T a9bed8fbdaca43e3832444f7f68e23a1 P 5899712e99ff4d88bb415b836db9967c: No bootstrap required, opened a new log
I20260618 22:27:40.774731 18696 ts_tablet_manager.cc:1397] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260618 22:27:40.774801 18695 ts_tablet_manager.cc:1397] T a9bed8fbdaca43e3832444f7f68e23a1 P 5899712e99ff4d88bb415b836db9967c: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260618 22:27:40.774847 18697 tablet_bootstrap.cc:492] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144: No bootstrap required, opened a new log
I20260618 22:27:40.774984 18697 ts_tablet_manager.cc:1397] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260618 22:27:40.775482 18695 raft_consensus.cc:348] T a9bed8fbdaca43e3832444f7f68e23a1 P 5899712e99ff4d88bb415b836db9967c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } } peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } }
I20260618 22:27:40.775667 18695 raft_consensus.cc:374] T a9bed8fbdaca43e3832444f7f68e23a1 P 5899712e99ff4d88bb415b836db9967c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:40.775527 18696 raft_consensus.cc:348] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } } peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } }
I20260618 22:27:40.775753 18695 raft_consensus.cc:729] T a9bed8fbdaca43e3832444f7f68e23a1 P 5899712e99ff4d88bb415b836db9967c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5899712e99ff4d88bb415b836db9967c, State: Initialized, Role: FOLLOWER
I20260618 22:27:40.775771 18696 raft_consensus.cc:374] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:40.775668 18697 raft_consensus.cc:348] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } } peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } }
I20260618 22:27:40.775847 18696 raft_consensus.cc:729] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4a2f32ef55149f2b75701234c139912, State: Initialized, Role: FOLLOWER
I20260618 22:27:40.775893 18697 raft_consensus.cc:374] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:40.775978 18697 raft_consensus.cc:729] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb54f92e5ac4441bed6f5bb6ea32144, State: Initialized, Role: FOLLOWER
I20260618 22:27:40.775969 18695 consensus_queue.cc:260] T a9bed8fbdaca43e3832444f7f68e23a1 P 5899712e99ff4d88bb415b836db9967c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } } peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } }
I20260618 22:27:40.776111 18696 consensus_queue.cc:260] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } } peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } }
I20260618 22:27:40.776194 18697 consensus_queue.cc:260] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } } peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } }
I20260618 22:27:40.776849 18695 ts_tablet_manager.cc:1428] T a9bed8fbdaca43e3832444f7f68e23a1 P 5899712e99ff4d88bb415b836db9967c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 22:27:40.777412 18697 ts_tablet_manager.cc:1428] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260618 22:27:40.781039 18696 ts_tablet_manager.cc:1428] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260618 22:27:40.781463 18705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.779035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.194:37569)
0618 22:27:40.781040 (+ 2005us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.781229 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1277,"thread_start_us":1102,"threads_started":1}
I20260618 22:27:40.783638 18460 heartbeater.cc:503] Master 127.10.242.253:33505 was elected leader, sending a full tablet report...
I20260618 22:27:40.783720 18601 heartbeater.cc:503] Master 127.10.242.253:33505 was elected leader, sending a full tablet report...
W20260618 22:27:40.785171 18461 tablet.cc:2369] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:27:40.786578 18702 raft_consensus.cc:482] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:27:40.786912 18702 raft_consensus.cc:504] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } } peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } }
I20260618 22:27:40.789192 18705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.782021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.194:52409)
0618 22:27:40.788896 (+ 6875us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.788994 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6777,"threads_started":1}
I20260618 22:27:40.781304 18704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.777844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.193:32989)
0618 22:27:40.780816 (+ 2972us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.780949 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2799,"thread_start_us":2677,"threads_started":1}
I20260618 22:27:40.789947 18702 leader_election.cc:288] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5899712e99ff4d88bb415b836db9967c (127.10.242.193:46667), 0fb54f92e5ac4441bed6f5bb6ea32144 (127.10.242.194:42531)
I20260618 22:27:40.790422 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.781132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.195:59239)
0618 22:27:40.789768 (+ 8636us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.789950 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8511,"thread_start_us":8397,"threads_started":1}
I20260618 22:27:40.792604 18709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.791588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.195:42917)
0618 22:27:40.792181 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.792391 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483}
I20260618 22:27:40.796864 18691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.789512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:60369 (local address 127.10.242.193:46667)
0618 22:27:40.789696 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:40.789711 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:40.790653 (+ 942us) server_negotiation.cc:407] Connection header received
0618 22:27:40.790971 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:40.790977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:40.791049 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:40.791109 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:40.791815 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.792749 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.795121 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.796108 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.796263 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:40.796337 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:40.796516 (+ 179us) server_negotiation.cc:299] Negotiation successful
0618 22:27:40.796640 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260618 22:27:40.796880 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.789043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:46667 (local address 127.10.242.195:60369)
0618 22:27:40.790564 (+ 1521us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.790580 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:40.790768 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:40.791293 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:40.791298 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:40.791322 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:40.791671 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:40.791692 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.792864 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.792869 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:40.794995 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:40.795011 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.796269 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.796274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:40.796386 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:40.796410 (+ 24us) client_negotiation.cc:769] Sending connection context
0618 22:27:40.796478 (+ 68us) client_negotiation.cc:240] Negotiation successful
0618 22:27:40.796567 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1398,"mutex_wait_us":3,"thread_start_us":1250,"threads_started":1}
I20260618 22:27:40.797712 18263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9bed8fbdaca43e3832444f7f68e23a1" candidate_uuid: "b4a2f32ef55149f2b75701234c139912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5899712e99ff4d88bb415b836db9967c" is_pre_election: true
I20260618 22:27:40.797891 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.791000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:45971 (local address 127.10.242.194:42531)
0618 22:27:40.791182 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:40.791189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:40.791513 (+ 324us) server_negotiation.cc:407] Connection header received
0618 22:27:40.791749 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:40.791754 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:40.791823 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:40.791881 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:40.792513 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.793431 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.795609 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.796714 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.796853 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:40.796923 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:40.797547 (+ 624us) server_negotiation.cc:299] Negotiation successful
0618 22:27:40.797680 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260618 22:27:40.797945 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.790695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:42531 (local address 127.10.242.195:45971)
0618 22:27:40.791423 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.791442 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:40.791592 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:40.792048 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:40.792056 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:40.792078 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:40.792372 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:40.792387 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.793549 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.793554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:40.795392 (+ 1838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:40.795401 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.797189 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.797195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:40.797361 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:40.797392 (+ 31us) client_negotiation.cc:769] Sending connection context
0618 22:27:40.797511 (+ 119us) client_negotiation.cc:240] Negotiation successful
0618 22:27:40.797643 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"mutex_wait_us":22,"thread_start_us":1210,"threads_started":1}
I20260618 22:27:40.798084 18263 raft_consensus.cc:2454] T a9bed8fbdaca43e3832444f7f68e23a1 P 5899712e99ff4d88bb415b836db9967c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4a2f32ef55149f2b75701234c139912 in term 0.
I20260618 22:27:40.798595 18413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9bed8fbdaca43e3832444f7f68e23a1" candidate_uuid: "b4a2f32ef55149f2b75701234c139912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" is_pre_election: true
I20260618 22:27:40.798797 18413 raft_consensus.cc:2454] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4a2f32ef55149f2b75701234c139912 in term 0.
I20260618 22:27:40.798827 18488 leader_election.cc:302] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5899712e99ff4d88bb415b836db9967c, b4a2f32ef55149f2b75701234c139912; no voters:
I20260618 22:27:40.799239 18702 raft_consensus.cc:2781] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:27:40.799361 18702 raft_consensus.cc:482] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:27:40.799455 18702 raft_consensus.cc:3037] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:40.801513 18702 raft_consensus.cc:504] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } } peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } }
I20260618 22:27:40.802022 18702 leader_election.cc:288] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [CANDIDATE]: Term 1 election: Requested vote from peers 5899712e99ff4d88bb415b836db9967c (127.10.242.193:46667), 0fb54f92e5ac4441bed6f5bb6ea32144 (127.10.242.194:42531)
I20260618 22:27:40.802492 18263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9bed8fbdaca43e3832444f7f68e23a1" candidate_uuid: "b4a2f32ef55149f2b75701234c139912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5899712e99ff4d88bb415b836db9967c"
I20260618 22:27:40.802507 18413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9bed8fbdaca43e3832444f7f68e23a1" candidate_uuid: "b4a2f32ef55149f2b75701234c139912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144"
I20260618 22:27:40.802665 18413 raft_consensus.cc:3037] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:40.802700 18263 raft_consensus.cc:3037] T a9bed8fbdaca43e3832444f7f68e23a1 P 5899712e99ff4d88bb415b836db9967c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:40.804719 18413 raft_consensus.cc:2454] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4a2f32ef55149f2b75701234c139912 in term 1.
I20260618 22:27:40.804776 18263 raft_consensus.cc:2454] T a9bed8fbdaca43e3832444f7f68e23a1 P 5899712e99ff4d88bb415b836db9967c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4a2f32ef55149f2b75701234c139912 in term 1.
I20260618 22:27:40.805060 18488 leader_election.cc:302] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb54f92e5ac4441bed6f5bb6ea32144, b4a2f32ef55149f2b75701234c139912; no voters:
I20260618 22:27:40.805358 18702 raft_consensus.cc:2781] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:40.805794 18702 raft_consensus.cc:686] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [term 1 LEADER]: Becoming Leader. State: Replica: b4a2f32ef55149f2b75701234c139912, State: Running, Role: LEADER
I20260618 22:27:40.806185 18702 consensus_queue.cc:237] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } } peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } }
I20260618 22:27:40.806726 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.806329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.195:34945)
0618 22:27:40.806483 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.806572 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260618 22:27:40.809720 17943 catalog_manager.cc:5495] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4a2f32ef55149f2b75701234c139912 (127.10.242.195). New cstate: current_term: 1 leader_uuid: "b4a2f32ef55149f2b75701234c139912" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4a2f32ef55149f2b75701234c139912" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 46165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 } health_report { overall_health: UNKNOWN } } }
I20260618 22:27:40.834259 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.829118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34138 (local address 127.10.242.195:46165)
0618 22:27:40.829303 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:40.829309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:40.829376 (+ 67us) server_negotiation.cc:407] Connection header received
0618 22:27:40.829590 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:40.829594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:40.829677 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:40.829831 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:27:40.830219 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.830908 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.832166 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.832584 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.832698 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:40.833164 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:27:40.833697 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:27:40.833840 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:40.833924 (+ 84us) server_negotiation.cc:299] Negotiation successful
0618 22:27:40.834005 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260618 22:27:40.838126 18413 raft_consensus.cc:1261] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144 [term 1 FOLLOWER]: Refusing update from remote peer b4a2f32ef55149f2b75701234c139912: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:40.838514 18263 raft_consensus.cc:1261] T a9bed8fbdaca43e3832444f7f68e23a1 P 5899712e99ff4d88bb415b836db9967c [term 1 FOLLOWER]: Refusing update from remote peer b4a2f32ef55149f2b75701234c139912: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:40.838811 18710 consensus_queue.cc:1035] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb54f92e5ac4441bed6f5bb6ea32144" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:40.839198 18702 consensus_queue.cc:1035] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5899712e99ff4d88bb415b836db9967c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:40.841430 18310 heartbeater.cc:503] Master 127.10.242.253:33505 was elected leader, sending a full tablet report...
I20260618 22:27:40.842159 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.840938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.194:44357)
0618 22:27:40.841801 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.841971 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":707}
I20260618 22:27:40.842624 18704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.841572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.193:40631)
0618 22:27:40.842440 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.842516 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":760}
I20260618 22:27:40.858870 18704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.858356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.193:54155)
0618 22:27:40.858558 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.858672 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260618 22:27:40.859972 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.859623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.194:48907)
0618 22:27:40.859753 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.859836 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260618 22:27:40.862219 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.861806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.195:50951)
0618 22:27:40.861982 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.862067 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260618 22:27:40.863443 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.862985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.10.242.195:42869)
0618 22:27:40.863183 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:40.863280 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260618 22:27:40.871861 17944 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:40.872085 17944 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:40.877717 17944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 468d5c19d3c54b7bbd0d7122ea722111: Sending DeleteTablet for 3 replicas of tablet a9bed8fbdaca43e3832444f7f68e23a1
I20260618 22:27:40.878767 18243 tablet_service.cc:1514] Processing DeleteTablet for tablet a9bed8fbdaca43e3832444f7f68e23a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:40 UTC) from {username='oryx', principal='oryx/127.10.242.253@KRBTEST.COM'} at 127.0.0.1:39448
I20260618 22:27:40.879415 18534 tablet_service.cc:1514] Processing DeleteTablet for tablet a9bed8fbdaca43e3832444f7f68e23a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:40 UTC) from {username='oryx', principal='oryx/127.10.242.253@KRBTEST.COM'} at 127.0.0.1:34124
I20260618 22:27:40.880157 18393 tablet_service.cc:1514] Processing DeleteTablet for tablet a9bed8fbdaca43e3832444f7f68e23a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:40 UTC) from {username='oryx', principal='oryx/127.10.242.253@KRBTEST.COM'} at 127.0.0.1:49696
I20260618 22:27:40.883549 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 18103
I20260618 22:27:40.892540 18724 tablet_replica.cc:330] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912: stopping tablet replica
I20260618 22:27:40.892794 18724 raft_consensus.cc:2229] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:27:40.893528 18724 raft_consensus.cc:2258] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:40.894821 18724 ts_tablet_manager.cc:1905] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:40.901500 18724 ts_tablet_manager.cc:1918] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:27:40.901652 18724 log.cc:1196] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/a9bed8fbdaca43e3832444f7f68e23a1
I20260618 22:27:40.902069 18724 ts_tablet_manager.cc:1939] T a9bed8fbdaca43e3832444f7f68e23a1 P b4a2f32ef55149f2b75701234c139912: Deleting consensus metadata
I20260618 22:27:40.902112 18725 tablet_replica.cc:330] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144: stopping tablet replica
I20260618 22:27:40.902313 18725 raft_consensus.cc:2229] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:40.902626 18725 raft_consensus.cc:2258] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:40.903540 17929 catalog_manager.cc:4841] TS b4a2f32ef55149f2b75701234c139912 (127.10.242.195:46165): tablet a9bed8fbdaca43e3832444f7f68e23a1 (table test-table [id=b4e3af3c3e3649b5aefac4fa6a70be16]) successfully deleted
I20260618 22:27:40.909857 18725 ts_tablet_manager.cc:1905] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:40.915052 18691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:40.884466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39464 (local address 127.10.242.193:46667)
0618 22:27:40.884649 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:40.884656 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:40.893796 (+ 9140us) server_negotiation.cc:407] Connection header received
0618 22:27:40.897285 (+ 3489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:40.897291 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:40.897398 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:40.897476 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:40.899367 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.900517 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.903098 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:40.903675 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:40.903771 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:40.907298 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:40.907318 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:40.907338 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:40.907380 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:40.911522 (+ 4142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:40.912405 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:40.912415 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:40.912419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:40.912571 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:40.913050 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:40.913055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:40.913059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:40.913547 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:40.913832 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:40.914404 (+ 572us) server_negotiation.cc:299] Negotiation successful
0618 22:27:40.914622 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:27:40.915207 18723 tablet_replica.cc:330] T a9bed8fbdaca43e3832444f7f68e23a1 P 5899712e99ff4d88bb415b836db9967c: stopping tablet replica
I20260618 22:27:40.915385 18723 raft_consensus.cc:2229] T a9bed8fbdaca43e3832444f7f68e23a1 P 5899712e99ff4d88bb415b836db9967c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:40.915658 18723 raft_consensus.cc:2258] T a9bed8fbdaca43e3832444f7f68e23a1 P 5899712e99ff4d88bb415b836db9967c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:40.916440 18303 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:40.917763 18725 ts_tablet_manager.cc:1918] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:27:40.917881 18725 log.cc:1196] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/a9bed8fbdaca43e3832444f7f68e23a1
I20260618 22:27:40.918264 18725 ts_tablet_manager.cc:1939] T a9bed8fbdaca43e3832444f7f68e23a1 P 0fb54f92e5ac4441bed6f5bb6ea32144: Deleting consensus metadata
I20260618 22:27:40.919256 17929 catalog_manager.cc:4841] TS 0fb54f92e5ac4441bed6f5bb6ea32144 (127.10.242.194:42531): tablet a9bed8fbdaca43e3832444f7f68e23a1 (table test-table [id=b4e3af3c3e3649b5aefac4fa6a70be16]) successfully deleted
I20260618 22:27:40.916864 18723 ts_tablet_manager.cc:1905] T a9bed8fbdaca43e3832444f7f68e23a1 P 5899712e99ff4d88bb415b836db9967c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:41.066162 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 18103
I20260618 22:27:41.085199 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 18322
I20260618 22:27:41.086108 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:41.085644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:46667 (local address 127.0.0.1:39466)
0618 22:27:41.085855 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:41.085943 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:46667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260618 22:27:41.086457 17929 catalog_manager.cc:4584] TS 5899712e99ff4d88bb415b836db9967c (127.10.242.193:46667): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a9bed8fbdaca43e3832444f7f68e23a1: Network error: Client connection negotiation failed: client connection to 127.10.242.193:46667: connect: Connection refused (error 111)
I20260618 22:27:41.098637 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:41.086116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49704 (local address 127.10.242.194:42531)
0618 22:27:41.086281 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:41.086287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:41.086425 (+ 138us) server_negotiation.cc:407] Connection header received
0618 22:27:41.086593 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:41.086599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:41.086692 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:41.086804 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:41.088020 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:41.088743 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:41.089993 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:41.090447 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:41.090564 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:41.093049 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:41.093063 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:41.093093 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:41.093133 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:41.095824 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:41.096455 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:41.096464 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:41.096468 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:41.096602 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:41.096951 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:41.096956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:41.096960 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:41.097466 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:41.097784 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:41.098122 (+ 338us) server_negotiation.cc:299] Negotiation successful
0618 22:27:41.098285 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:27:41.099282 18453 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:41.147194 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:41.146694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:46667 (local address 127.0.0.1:39480)
0618 22:27:41.146923 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:41.147032 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:46667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260618 22:27:41.148263 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:41.147890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:46667 (local address 127.0.0.1:39496)
0618 22:27:41.148057 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:41.148145 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:46667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260618 22:27:41.216589 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:41.216178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:46667 (local address 127.0.0.1:39500)
0618 22:27:41.216368 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:41.216449 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:46667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260618 22:27:41.217553 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:41.217215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:46667 (local address 127.0.0.1:39508)
0618 22:27:41.217370 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:41.217441 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:46667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260618 22:27:41.263464 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 18322
I20260618 22:27:41.272504 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:41.271925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.0.0.1:34506)
0618 22:27:41.272195 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:41.272302 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260618 22:27:41.273933 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:41.273384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:34635 (local address 127.0.0.1:34516)
0618 22:27:41.273701 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:41.273790 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:34635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
I20260618 22:27:41.278345 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 18471
I20260618 22:27:41.293215 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:41.279095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34144 (local address 127.10.242.195:46165)
0618 22:27:41.279251 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:41.279255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:41.279297 (+ 42us) server_negotiation.cc:407] Connection header received
0618 22:27:41.279566 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:41.279572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:41.279637 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:41.279766 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:41.281530 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:41.282302 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:41.283958 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:41.284408 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:41.284501 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:41.287270 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:41.287282 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:41.287297 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:41.287332 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:41.290479 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:41.291128 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:41.291135 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:41.291138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:41.291281 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:41.291642 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:41.291645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:41.291647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:41.291999 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:41.292317 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:41.292604 (+ 287us) server_negotiation.cc:299] Negotiation successful
0618 22:27:41.292804 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:27:41.293850 18594 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:41.319419 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:41.318958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:46667 (local address 127.0.0.1:39510)
0618 22:27:41.319164 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:41.319257 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:46667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260618 22:27:41.320314 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:41.320007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:46667 (local address 127.0.0.1:39514)
0618 22:27:41.320145 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:41.320208 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:46667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260618 22:27:41.422470 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 18471
I20260618 22:27:41.438676 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 17806
I20260618 22:27:41.439723 17974 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:41.557556 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:41.557060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:46667 (local address 127.0.0.1:39518)
0618 22:27:41.557267 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:41.557365 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:46667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260618 22:27:41.557600 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 17806
I20260618 22:27:41.573614 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 17908
I20260618 22:27:41.574666 18161 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:41.685156 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 17908
2026-06-18T22:27:41Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9213 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:27:41 dist-test-slave-0956 krb5kdc[18734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:27:41 dist-test-slave-0956 krb5kdc[18734](info): set up 2 sockets
Jun 18 22:27:41 dist-test-slave-0956 krb5kdc[18734](info): commencing operation
krb5kdc: starting...
W20260618 22:27:42.682049 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.958s user 0.006s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:27:42 dist-test-slave-0956 krb5kdc[18734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:27:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:27:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.254.keytab.
Entry for principal oryx/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.254.keytab.
I20260618 22:27:42.882452 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:38265
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.254.keytab
--principal=oryx/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:38265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:43.194229 18747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:43.194522 18747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:43.194629 18747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:43.201916 18747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:27:43.202025 18747 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260618 22:27:43.202064 18747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:43.202126 18747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:43.202174 18747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:27:43.202239 18747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:27:43.212210 18747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:38265
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.254.keytab
--principal=oryx/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:38265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:43.213482 18747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:43.214740 18747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:43.225740 18753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:43.225800 18752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:43.225822 18755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:43.226670 18747 server_base.cc:1034] running on GCE node
I20260618 22:27:43.227383 18747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:43.228715 18747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:43.229933 18747 hybrid_clock.cc:648] HybridClock initialized: now 1781821663229877 us; error 30 us; skew 500 ppm
I20260618 22:27:43.230242 18747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:43 dist-test-slave-0956 krb5kdc[18734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821663, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:43.233253 18747 init.cc:377] Logged in from keytab as oryx/127.10.242.254@KRBTEST.COM (short username oryx)
I20260618 22:27:43.234696 18747 webserver.cc:460] Webserver started at http://127.10.242.254:46499/ using document root <none> and password file <none>
I20260618 22:27:43.235354 18747 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:43.235448 18747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:43.235740 18747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:43.238569 18747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "7b73fef7fbc143a7b5f98f2ad8c99442"
format_stamp: "Formatted at 2026-06-18 22:27:43 on dist-test-slave-0956"
server_key: "dc5f0b1e9e5ca2911afa668c2ca94345"
server_key_iv: "f4cad6c43895485d4ed43b7fcdf50651"
server_key_version: "encryptionkey@0"
I20260618 22:27:43.239305 18747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "7b73fef7fbc143a7b5f98f2ad8c99442"
format_stamp: "Formatted at 2026-06-18 22:27:43 on dist-test-slave-0956"
server_key: "dc5f0b1e9e5ca2911afa668c2ca94345"
server_key_iv: "f4cad6c43895485d4ed43b7fcdf50651"
server_key_version: "encryptionkey@0"
I20260618 22:27:43.244202 18747 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260618 22:27:43.247617 18762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:43.248939 18747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:27:43.249121 18747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "7b73fef7fbc143a7b5f98f2ad8c99442"
format_stamp: "Formatted at 2026-06-18 22:27:43 on dist-test-slave-0956"
server_key: "dc5f0b1e9e5ca2911afa668c2ca94345"
server_key_iv: "f4cad6c43895485d4ed43b7fcdf50651"
server_key_version: "encryptionkey@0"
I20260618 22:27:43.249326 18747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:43.263564 18747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:43.267403 18747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:43.267653 18747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:43.287412 18747 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:38265
I20260618 22:27:43.287420 18823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:38265 every 8 connection(s)
I20260618 22:27:43.288933 18747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:27:43.291721 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 18747
I20260618 22:27:43.291930 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:27:43.292501 11211 external_mini_cluster.cc:1428] Setting key f6752134b47688bb30d04ca60683696f
I20260618 22:27:43.292611 18824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:43.305922 18824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442: Bootstrap starting.
Jun 18 22:27:43 dist-test-slave-0956 krb5kdc[18734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM
I20260618 22:27:43.309521 18824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:43.310817 18824 log.cc:824] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:43.313483 18824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442: No bootstrap required, opened a new log
I20260618 22:27:43.314595 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:43.294336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59012 (local address 127.10.242.254:38265)
0618 22:27:43.295113 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:43.295124 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:43.295166 (+ 42us) server_negotiation.cc:407] Connection header received
0618 22:27:43.296921 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:43.296943 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:43.297363 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:43.297682 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:43.299353 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:43.300577 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:43.302230 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:43.302941 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:43.303073 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:43.307308 (+ 4235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:43.307319 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:43.307336 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:43.307391 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:43.310928 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:43.311536 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:43.311543 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:43.311548 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:43.311703 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:43.312041 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:43.312045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:43.312047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:43.312558 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:43.312814 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:43.313252 (+ 438us) server_negotiation.cc:299] Negotiation successful
0618 22:27:43.313648 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":342,"threads_started":1}
I20260618 22:27:43.321439 18824 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b73fef7fbc143a7b5f98f2ad8c99442" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 38265 } }
I20260618 22:27:43.321805 18824 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:43.321918 18824 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b73fef7fbc143a7b5f98f2ad8c99442, State: Initialized, Role: FOLLOWER
I20260618 22:27:43.322503 18824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b73fef7fbc143a7b5f98f2ad8c99442" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 38265 } }
I20260618 22:27:43.322741 18824 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:27:43.322870 18824 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:27:43.323024 18824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:43.325448 18824 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b73fef7fbc143a7b5f98f2ad8c99442" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 38265 } }
I20260618 22:27:43.326009 18824 leader_election.cc:302] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b73fef7fbc143a7b5f98f2ad8c99442; no voters:
I20260618 22:27:43.326548 18824 leader_election.cc:288] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:27:43.326654 18829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:43.327287 18829 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442 [term 1 LEADER]: Becoming Leader. State: Replica: 7b73fef7fbc143a7b5f98f2ad8c99442, State: Running, Role: LEADER
I20260618 22:27:43.327764 18829 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b73fef7fbc143a7b5f98f2ad8c99442" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 38265 } }
I20260618 22:27:43.328554 18824 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:27:43.330299 18831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b73fef7fbc143a7b5f98f2ad8c99442. Latest consensus state: current_term: 1 leader_uuid: "7b73fef7fbc143a7b5f98f2ad8c99442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b73fef7fbc143a7b5f98f2ad8c99442" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 38265 } } }
I20260618 22:27:43.330624 18830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b73fef7fbc143a7b5f98f2ad8c99442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b73fef7fbc143a7b5f98f2ad8c99442" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 38265 } } }
I20260618 22:27:43.330824 18831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:43.330893 18830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442 [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:43.333801 18836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:27:43.338861 18836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:27:43.347882 18836 catalog_manager.cc:1347] Generated new cluster ID: 93c0b999c4404e1f94f714a2ecb07352
I20260618 22:27:43.347992 18836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:27:43.366819 18836 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:27:43.368309 18836 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:27:43.376959 18836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442: Generated new TSK 0
I20260618 22:27:43.377772 18836 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.193.keytab.
Entry for principal oryx/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.193.keytab.
I20260618 22:27:43.428896 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:38265
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.193.keytab
--principal=oryx/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:43.750211 18850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:43.750499 18850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:43.750633 18850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:43.757915 18850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:43.758044 18850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:43.758162 18850 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:27:43.758276 18850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:27:43.767998 18850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.193.keytab
--principal=oryx/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.242.254:38265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:43.769377 18850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:43.770684 18850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:43.783754 18855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:43.783664 18858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:43.783643 18856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:43.784139 18850 server_base.cc:1034] running on GCE node
I20260618 22:27:43.784914 18850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:43.785882 18850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:43.787092 18850 hybrid_clock.cc:648] HybridClock initialized: now 1781821663787023 us; error 49 us; skew 500 ppm
I20260618 22:27:43.787384 18850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:43 dist-test-slave-0956 krb5kdc[18734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821663, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:43.790431 18850 init.cc:377] Logged in from keytab as oryx/127.10.242.193@KRBTEST.COM (short username oryx)
I20260618 22:27:43.791857 18850 webserver.cc:460] Webserver started at http://127.10.242.193:43047/ using document root <none> and password file <none>
I20260618 22:27:43.792510 18850 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:43.792591 18850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:43.792856 18850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:43.795758 18850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7"
format_stamp: "Formatted at 2026-06-18 22:27:43 on dist-test-slave-0956"
server_key: "ff6be5c29a013c4e38ae0eddb9dfe1dc"
server_key_iv: "78312183cacd098d00f87d1471134139"
server_key_version: "encryptionkey@0"
I20260618 22:27:43.796483 18850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7"
format_stamp: "Formatted at 2026-06-18 22:27:43 on dist-test-slave-0956"
server_key: "ff6be5c29a013c4e38ae0eddb9dfe1dc"
server_key_iv: "78312183cacd098d00f87d1471134139"
server_key_version: "encryptionkey@0"
I20260618 22:27:43.801651 18850 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260618 22:27:43.805243 18865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:43.806655 18850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:27:43.806826 18850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7"
format_stamp: "Formatted at 2026-06-18 22:27:43 on dist-test-slave-0956"
server_key: "ff6be5c29a013c4e38ae0eddb9dfe1dc"
server_key_iv: "78312183cacd098d00f87d1471134139"
server_key_version: "encryptionkey@0"
I20260618 22:27:43.807032 18850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:43.834643 18850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:43.838790 18850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:43.839031 18850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:43.841601 18850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:43.841678 18850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:43.841800 18850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:43.841872 18850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:27:43 dist-test-slave-0956 krb5kdc[18734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821663, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM
I20260618 22:27:43.866109 18874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:43.842510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:43071 (local address 127.10.242.254:38265)
0618 22:27:43.842925 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:43.842930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:43.844316 (+ 1386us) server_negotiation.cc:407] Connection header received
0618 22:27:43.846602 (+ 2286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:43.846609 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:43.846704 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:43.846832 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:43.849126 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:43.850124 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:43.853157 (+ 3033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:43.853837 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:43.853969 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:43.858880 (+ 4911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:43.858887 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:43.858893 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:43.858939 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:43.862275 (+ 3336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:43.863271 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:43.863277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:43.863281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:43.863411 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:43.864000 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:43.864006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:43.864010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:43.864357 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:43.864494 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:43.865473 (+ 979us) server_negotiation.cc:299] Negotiation successful
0618 22:27:43.865705 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":235,"threads_started":1}
I20260618 22:27:43.867189 18875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:43.843208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:38265 (local address 127.10.242.193:43071)
0618 22:27:43.844135 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:43.844178 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:43.846215 (+ 2037us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:43.847155 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:43.847173 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:43.847987 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:43.848908 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:43.848936 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:43.850336 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:43.850341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:43.852902 (+ 2561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:43.852919 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:43.854131 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:43.854145 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:43.854480 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:43.855353 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:43.855381 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:43.858616 (+ 3235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:43.862486 (+ 3870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:43.862498 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:43.862522 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:43.863153 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:43.863542 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:43.863549 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:43.863553 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:43.863883 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:43.864505 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:43.864513 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:43.865037 (+ 524us) client_negotiation.cc:769] Sending connection context
0618 22:27:43.865389 (+ 352us) client_negotiation.cc:240] Negotiation successful
0618 22:27:43.865985 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":435,"threads_started":1}
I20260618 22:27:43.887984 18850 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:43591
I20260618 22:27:43.888049 18985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:43591 every 8 connection(s)
I20260618 22:27:43.889401 18850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:27:43.891516 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 18850
I20260618 22:27:43.891724 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:27:43.891777 18986 heartbeater.cc:347] Connected to a master server at 127.10.242.254:38265
I20260618 22:27:43.892194 18874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:43.876535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:36191 (local address 127.10.242.254:38265)
0618 22:27:43.876716 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:43.876723 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:43.877889 (+ 1166us) server_negotiation.cc:407] Connection header received
0618 22:27:43.878056 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:43.878062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:43.878140 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:43.878242 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:43.880110 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:43.880837 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:43.883056 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:43.883664 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:43.883769 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:43.886630 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:43.886635 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:43.886638 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:43.886670 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:43.889376 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:43.890101 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:43.890108 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:43.890112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:43.890290 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:43.890700 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:43.890705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:43.890709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:43.891014 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:43.891183 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:43.891679 (+ 496us) server_negotiation.cc:299] Negotiation successful
0618 22:27:43.891896 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:27:43.892395 18955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:43.877001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:38265 (local address 127.10.242.193:36191)
0618 22:27:43.877774 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:43.877798 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:43.877939 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:43.878432 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:43.878437 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:43.879336 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:43.879903 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:43.879923 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:43.881027 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:43.881032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:43.882864 (+ 1832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:43.882882 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:43.884105 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:43.884112 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:43.884314 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:43.885172 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:43.885195 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:43.886430 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:43.889496 (+ 3066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:43.889502 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:43.889506 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:43.889938 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:43.890408 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:43.890412 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:43.890415 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:43.890574 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:43.891248 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:43.891252 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:43.891427 (+ 175us) client_negotiation.cc:769] Sending connection context
0618 22:27:43.891573 (+ 146us) client_negotiation.cc:240] Negotiation successful
0618 22:27:43.891926 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":507,"threads_started":1}
I20260618 22:27:43.892504 11211 external_mini_cluster.cc:1428] Setting key d541cfe8b02b1664128424f793f5cbf6
I20260618 22:27:43.892020 18986 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:43.893319 18986 heartbeater.cc:511] Master 127.10.242.254:38265 requested a full tablet report, sending...
I20260618 22:27:43.895390 18778 ts_manager.cc:194] Registered new tserver with Master: bdde0480a48d43c99b6fd50ca6bd9ce7 (127.10.242.193:43591)
I20260618 22:27:43.896862 18778 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:43071
WARNING: no policy specified for oryx/127.10.242.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.194.keytab.
Entry for principal oryx/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.194.keytab.
I20260618 22:27:43.940876 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:38265
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.194.keytab
--principal=oryx/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:44.261237 18990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:44.261540 18990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:44.261678 18990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:44.268889 18990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:44.269023 18990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:44.269166 18990 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:27:44.269279 18990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:27:44.279198 18990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.194.keytab
--principal=oryx/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.242.254:38265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:44.280411 18990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:44.281702 18990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:44.296290 18998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:44.296241 18996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:44.296619 18995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:44.298475 18990 server_base.cc:1034] running on GCE node
I20260618 22:27:44.299060 18990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:44.299983 18990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:44.301180 18990 hybrid_clock.cc:648] HybridClock initialized: now 1781821664301104 us; error 52 us; skew 500 ppm
I20260618 22:27:44.301466 18990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:44 dist-test-slave-0956 krb5kdc[18734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821664, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:44.304411 18990 init.cc:377] Logged in from keytab as oryx/127.10.242.194@KRBTEST.COM (short username oryx)
I20260618 22:27:44.305835 18990 webserver.cc:460] Webserver started at http://127.10.242.194:34401/ using document root <none> and password file <none>
I20260618 22:27:44.306514 18990 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:44.306605 18990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:44.306886 18990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:44.309727 18990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "cfd7a2d3199942488cace531d1903c76"
format_stamp: "Formatted at 2026-06-18 22:27:44 on dist-test-slave-0956"
server_key: "0d06d463e4bd524ac4341f1c4a2655ba"
server_key_iv: "d892901cba35f28cce7464d9ade8ccb3"
server_key_version: "encryptionkey@0"
I20260618 22:27:44.310526 18990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "cfd7a2d3199942488cace531d1903c76"
format_stamp: "Formatted at 2026-06-18 22:27:44 on dist-test-slave-0956"
server_key: "0d06d463e4bd524ac4341f1c4a2655ba"
server_key_iv: "d892901cba35f28cce7464d9ade8ccb3"
server_key_version: "encryptionkey@0"
I20260618 22:27:44.315410 18990 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260618 22:27:44.318727 19005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:44.319950 18990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:27:44.320088 18990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "cfd7a2d3199942488cace531d1903c76"
format_stamp: "Formatted at 2026-06-18 22:27:44 on dist-test-slave-0956"
server_key: "0d06d463e4bd524ac4341f1c4a2655ba"
server_key_iv: "d892901cba35f28cce7464d9ade8ccb3"
server_key_version: "encryptionkey@0"
I20260618 22:27:44.320277 18990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:44.342803 18990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:44.346870 18990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:44.347107 18990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:44.349674 18990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:44.349758 18990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:44.349881 18990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:44.349936 18990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:27:44 dist-test-slave-0956 krb5kdc[18734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821664, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM
I20260618 22:27:44.372067 18874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:44.350389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:40169 (local address 127.10.242.254:38265)
0618 22:27:44.350587 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:44.350593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:44.352022 (+ 1429us) server_negotiation.cc:407] Connection header received
0618 22:27:44.354501 (+ 2479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:44.354508 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:44.354637 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:44.354699 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:44.357199 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.358391 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.360065 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.360753 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.360884 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:44.365318 (+ 4434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:44.365324 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:44.365329 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:44.365371 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:44.368609 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:44.369660 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:44.369666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:44.369669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:44.369831 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:44.370335 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:44.370341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:44.370345 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:44.370648 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:44.370789 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:44.371497 (+ 708us) server_negotiation.cc:299] Negotiation successful
0618 22:27:44.371710 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260618 22:27:44.373041 19014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:44.351010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:38265 (local address 127.10.242.194:40169)
0618 22:27:44.351853 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:44.351907 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:44.354128 (+ 2221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:44.355240 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:44.355251 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:44.356056 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:44.356969 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:44.356997 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.358571 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.358576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:44.359936 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:44.359949 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.361029 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.361038 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:44.361354 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:44.362071 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:44.362094 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:44.365053 (+ 2959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:44.368756 (+ 3703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:44.368765 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:44.368786 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:44.369494 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:44.369943 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:44.369948 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:44.369952 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:44.370208 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:44.370769 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:44.370777 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:44.371137 (+ 360us) client_negotiation.cc:769] Sending connection context
0618 22:27:44.371370 (+ 233us) client_negotiation.cc:240] Negotiation successful
0618 22:27:44.371897 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":344,"threads_started":1}
I20260618 22:27:44.392100 18874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:44.378188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:36261 (local address 127.10.242.254:38265)
0618 22:27:44.378358 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:44.378364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:44.378986 (+ 622us) server_negotiation.cc:407] Connection header received
0618 22:27:44.379150 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:44.379153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:44.379213 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:44.379286 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:44.380419 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.381183 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.383144 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.383791 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.383915 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:44.386339 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:44.386345 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:44.386350 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:44.386389 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:44.389479 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:44.390238 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:44.390244 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:44.390248 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:44.390367 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:44.390701 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:44.390707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:44.390711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:44.391074 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:44.391199 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:44.391562 (+ 363us) server_negotiation.cc:299] Negotiation successful
0618 22:27:44.391766 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260618 22:27:44.392206 19069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:44.378232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:38265 (local address 127.10.242.194:36261)
0618 22:27:44.378876 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:44.378897 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:44.379039 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:44.379329 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:44.379335 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:44.379906 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:44.380276 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:44.380288 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.381330 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.381333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:44.382937 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:44.382949 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.383989 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.383995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:44.384234 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:44.384898 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:44.384929 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:44.386108 (+ 1179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:44.389606 (+ 3498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:44.389615 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:44.389619 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:44.390131 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:44.390468 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:44.390473 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:44.390477 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:44.390598 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:44.391177 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:44.391182 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:44.391351 (+ 169us) client_negotiation.cc:769] Sending connection context
0618 22:27:44.391465 (+ 114us) client_negotiation.cc:240] Negotiation successful
0618 22:27:44.391778 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":316,"threads_started":1}
I20260618 22:27:44.400089 18990 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:37169
I20260618 22:27:44.400127 19124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:37169 every 8 connection(s)
I20260618 22:27:44.401269 18990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:27:44.403304 19125 heartbeater.cc:347] Connected to a master server at 127.10.242.254:38265
I20260618 22:27:44.403496 19125 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:44.404146 19125 heartbeater.cc:511] Master 127.10.242.254:38265 requested a full tablet report, sending...
I20260618 22:27:44.405443 18778 ts_manager.cc:194] Registered new tserver with Master: cfd7a2d3199942488cace531d1903c76 (127.10.242.194:37169)
I20260618 22:27:44.406208 18778 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:40169
I20260618 22:27:44.411603 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 18990
I20260618 22:27:44.411789 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:27:44.412150 11211 external_mini_cluster.cc:1428] Setting key 272cfe49ce977860ee1e3536600c7f90
WARNING: no policy specified for oryx/127.10.242.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.195.keytab.
Entry for principal oryx/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.195.keytab.
I20260618 22:27:44.456657 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:38265
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.195.keytab
--principal=oryx/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:44.772120 19129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:44.772435 19129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:44.772578 19129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:44.780045 19129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:44.780177 19129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:44.780301 19129 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:27:44.780408 19129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:27:44.790257 19129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.195.keytab
--principal=oryx/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.242.254:38265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:44.791531 19129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:44.792819 19129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:44.805757 19137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:44.805691 19135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:44.805665 19134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:44.806576 19129 server_base.cc:1034] running on GCE node
I20260618 22:27:44.807160 19129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:44.808089 19129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:44.809319 19129 hybrid_clock.cc:648] HybridClock initialized: now 1781821664809246 us; error 40 us; skew 500 ppm
I20260618 22:27:44.809599 19129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:44 dist-test-slave-0956 krb5kdc[18734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821664, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:44.812665 19129 init.cc:377] Logged in from keytab as oryx/127.10.242.195@KRBTEST.COM (short username oryx)
I20260618 22:27:44.814121 19129 webserver.cc:460] Webserver started at http://127.10.242.195:42849/ using document root <none> and password file <none>
I20260618 22:27:44.814914 19129 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:44.815011 19129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:44.815310 19129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:44.818114 19129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "ba3586b59246465dbef57edb8472d07f"
format_stamp: "Formatted at 2026-06-18 22:27:44 on dist-test-slave-0956"
server_key: "e2189100fad0bb571a559a374425ecb3"
server_key_iv: "3f921b531aab84e92db6197b5d7b7964"
server_key_version: "encryptionkey@0"
I20260618 22:27:44.818849 19129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "ba3586b59246465dbef57edb8472d07f"
format_stamp: "Formatted at 2026-06-18 22:27:44 on dist-test-slave-0956"
server_key: "e2189100fad0bb571a559a374425ecb3"
server_key_iv: "3f921b531aab84e92db6197b5d7b7964"
server_key_version: "encryptionkey@0"
I20260618 22:27:44.823796 19129 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:27:44.827095 19144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:44.828387 19129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:27:44.828541 19129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "ba3586b59246465dbef57edb8472d07f"
format_stamp: "Formatted at 2026-06-18 22:27:44 on dist-test-slave-0956"
server_key: "e2189100fad0bb571a559a374425ecb3"
server_key_iv: "3f921b531aab84e92db6197b5d7b7964"
server_key_version: "encryptionkey@0"
I20260618 22:27:44.828748 19129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:44.857668 19129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:44.861613 19129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:44.861860 19129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:44.864362 19129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:44.864444 19129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:44.864558 19129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:44.864610 19129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:27:44 dist-test-slave-0956 krb5kdc[18734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821664, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM
I20260618 22:27:44.888675 18874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:44.865075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:33391 (local address 127.10.242.254:38265)
0618 22:27:44.865292 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:44.865298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:44.866724 (+ 1426us) server_negotiation.cc:407] Connection header received
0618 22:27:44.869654 (+ 2930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:44.869661 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:44.869752 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:44.869888 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:44.872332 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.873274 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.876015 (+ 2741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.876663 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.876797 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:44.881800 (+ 5003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:44.881808 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:44.881813 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:44.881856 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:44.885293 (+ 3437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:44.886154 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:44.886161 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:44.886165 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:44.886340 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:44.886777 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:44.886782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:44.886786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:44.887147 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:44.887311 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:44.888092 (+ 781us) server_negotiation.cc:299] Negotiation successful
0618 22:27:44.888324 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:27:44.889883 19153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:44.865720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:38265 (local address 127.10.242.195:33391)
0618 22:27:44.866540 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:44.866594 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:44.869241 (+ 2647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:44.870168 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:44.870183 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:44.871062 (+ 879us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:44.872121 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:44.872153 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.873447 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.873457 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:44.875843 (+ 2386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:44.875859 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.876919 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.876927 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:44.877277 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:44.878047 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:44.878083 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:44.881500 (+ 3417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:44.885457 (+ 3957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:44.885467 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:44.885498 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:44.886028 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:44.886455 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:44.886459 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:44.886461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:44.886684 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:44.887297 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:44.887303 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:44.887653 (+ 350us) client_negotiation.cc:769] Sending connection context
0618 22:27:44.887915 (+ 262us) client_negotiation.cc:240] Negotiation successful
0618 22:27:44.888455 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":308,"threads_started":1}
I20260618 22:27:44.899459 18986 heartbeater.cc:503] Master 127.10.242.254:38265 was elected leader, sending a full tablet report...
I20260618 22:27:44.910866 18874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:44.897425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:54547 (local address 127.10.242.254:38265)
0618 22:27:44.897581 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:44.897585 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:44.898148 (+ 563us) server_negotiation.cc:407] Connection header received
0618 22:27:44.898293 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:44.898300 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:44.898398 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:44.898495 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:44.899359 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.900162 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.902148 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.902571 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.902663 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:44.904955 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:44.904961 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:44.904965 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:44.905002 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:44.908289 (+ 3287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:44.908882 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:44.908888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:44.908892 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:44.909029 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:44.909429 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:44.909434 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:44.909437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:44.909789 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:44.909928 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:44.910429 (+ 501us) server_negotiation.cc:299] Negotiation successful
0618 22:27:44.910584 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:27:44.911975 19223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:44.897510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:38265 (local address 127.10.242.195:54547)
0618 22:27:44.898057 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:44.898076 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:44.898181 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:44.898507 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:44.898510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:44.898952 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:44.899221 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:44.899230 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.900293 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.900299 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:44.902009 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:44.902026 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.902746 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.902752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:44.902938 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:44.903538 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:44.903558 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:44.904738 (+ 1180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:44.908411 (+ 3673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:44.908415 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:44.908418 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:44.908790 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:44.909168 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:44.909173 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:44.909177 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:44.909328 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:44.909933 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:44.909939 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:44.910166 (+ 227us) client_negotiation.cc:769] Sending connection context
0618 22:27:44.910322 (+ 156us) client_negotiation.cc:240] Negotiation successful
0618 22:27:44.911399 (+ 1077us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":302,"threads_started":1}
I20260618 22:27:44.919550 19129 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:36539
I20260618 22:27:44.919623 19263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:36539 every 8 connection(s)
I20260618 22:27:44.920701 19129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:27:44.922657 19264 heartbeater.cc:347] Connected to a master server at 127.10.242.254:38265
I20260618 22:27:44.922870 19264 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:44.923537 19264 heartbeater.cc:511] Master 127.10.242.254:38265 requested a full tablet report, sending...
I20260618 22:27:44.924777 18778 ts_manager.cc:194] Registered new tserver with Master: ba3586b59246465dbef57edb8472d07f (127.10.242.195:36539)
I20260618 22:27:44.925606 18778 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:33391
I20260618 22:27:44.926501 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 19129
I20260618 22:27:44.926654 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:27:44.927034 11211 external_mini_cluster.cc:1428] Setting key c832bb2ad0fa917d307fb01d6e0fc699
I20260618 22:27:44.929767 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 18 22:27:44 dist-test-slave-0956 krb5kdc[18734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821664, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 18 22:27:44 dist-test-slave-0956 krb5kdc[18734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:44 dist-test-slave-0956 krb5kdc[18734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
W20260618 22:27:44.975766 18874 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:44.965879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59026 (local address 127.10.242.254:38265)
0618 22:27:44.966101 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:44.966107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:44.966265 (+ 158us) server_negotiation.cc:407] Connection header received
0618 22:27:44.966412 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:44.966416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:44.966479 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:44.966563 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:44.967965 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.968689 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.970019 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.970417 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.970508 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:44.975517 (+ 5009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59026: BlockingRecv error: recv got EOF from 127.0.0.1:59026 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Jun 18 22:27:44 dist-test-slave-0956 krb5kdc[18734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821664, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM
I20260618 22:27:44.991169 18874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:44.978641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59038 (local address 127.10.242.254:38265)
0618 22:27:44.978836 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:44.978842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:44.979090 (+ 248us) server_negotiation.cc:407] Connection header received
0618 22:27:44.979282 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:44.979287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:44.979360 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:44.979439 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:44.980672 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.981450 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.982647 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:44.983068 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:44.983221 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:44.986688 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:44.986693 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:44.986697 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:44.986733 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:44.989112 (+ 2379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:44.989750 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:44.989754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:44.989758 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:44.989868 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:44.990145 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:44.990148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:44.990150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:44.990409 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:44.990628 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:44.990792 (+ 164us) server_negotiation.cc:299] Negotiation successful
0618 22:27:44.990948 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:27:44.995149 18778 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59038:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:27:45.002259 18778 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:27:45.027750 19283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.014816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:43591 (local address 127.0.0.1:51838)
0618 22:27:45.017213 (+ 2397us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:45.017234 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:45.017439 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:45.018358 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:45.018363 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:45.018395 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:45.019304 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.019317 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.021163 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.021168 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.024372 (+ 3204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.024397 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.026705 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.026711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.026919 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.026975 (+ 56us) client_negotiation.cc:769] Sending connection context
0618 22:27:45.027116 (+ 141us) client_negotiation.cc:240] Negotiation successful
0618 22:27:45.027320 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2286,"mutex_wait_us":502,"thread_start_us":2124,"threads_started":1}
I20260618 22:27:45.028482 19285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.015278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51838 (local address 127.10.242.193:43591)
0618 22:27:45.016760 (+ 1482us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:45.016769 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:45.017338 (+ 569us) server_negotiation.cc:407] Connection header received
0618 22:27:45.017615 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:45.017623 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:45.017884 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:45.018063 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:45.019518 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.020963 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.024600 (+ 3637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.026190 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.026343 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.026531 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:45.028018 (+ 1487us) server_negotiation.cc:299] Negotiation successful
0618 22:27:45.028201 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1343,"thread_start_us":1234,"threads_started":1}
I20260618 22:27:45.029004 19284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.014971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37414 (local address 127.10.242.195:36539)
0618 22:27:45.016368 (+ 1397us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:45.016379 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:45.016911 (+ 532us) server_negotiation.cc:407] Connection header received
0618 22:27:45.017388 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:45.017398 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:45.017674 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:45.017858 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:45.018971 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.020713 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.024686 (+ 3973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.027592 (+ 2906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.027757 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.027941 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:45.028523 (+ 582us) server_negotiation.cc:299] Negotiation successful
0618 22:27:45.028691 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1194,"thread_start_us":1037,"threads_started":1}
I20260618 22:27:45.029763 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.014635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:36539 (local address 127.0.0.1:37414)
0618 22:27:45.016758 (+ 2123us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:45.016808 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:45.017117 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:45.018235 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:45.018242 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:45.018268 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:45.018698 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.018720 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.020883 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.020888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.024504 (+ 3616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.024516 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.028131 (+ 3615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.028136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.028314 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.028347 (+ 33us) client_negotiation.cc:769] Sending connection context
0618 22:27:45.029226 (+ 879us) client_negotiation.cc:240] Negotiation successful
0618 22:27:45.029409 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1853,"mutex_wait_us":21,"thread_start_us":1721,"threads_started":1}
I20260618 22:27:45.033278 18916 tablet_service.cc:1467] Processing CreateTablet for tablet c86a9596ff6449b3948aa286e181f7a9 (DEFAULT_TABLE table=test-table [id=ccea0b6733f64b2eb863c33439505b94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:45.034533 18916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c86a9596ff6449b3948aa286e181f7a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:45.034660 19194 tablet_service.cc:1467] Processing CreateTablet for tablet c86a9596ff6449b3948aa286e181f7a9 (DEFAULT_TABLE table=test-table [id=ccea0b6733f64b2eb863c33439505b94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:45.036017 19194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c86a9596ff6449b3948aa286e181f7a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:45.039124 19283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.017213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:37169 (local address 127.0.0.1:56342)
0618 22:27:45.028925 (+ 11712us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:45.028945 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:45.029259 (+ 314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:45.030354 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:45.030359 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:45.030387 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:45.030563 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.030576 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.033184 (+ 2608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.033190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.035087 (+ 1897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.035105 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.036854 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.036861 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.038517 (+ 1656us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.038556 (+ 39us) client_negotiation.cc:769] Sending connection context
0618 22:27:45.038659 (+ 103us) client_negotiation.cc:240] Negotiation successful
0618 22:27:45.038817 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11614,"thread_start_us":13463,"threads_started":1}
I20260618 22:27:45.039218 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.017473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56342 (local address 127.10.242.194:37169)
0618 22:27:45.029745 (+ 12272us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:45.029754 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:45.029784 (+ 30us) server_negotiation.cc:407] Connection header received
0618 22:27:45.029912 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:45.029920 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:45.030164 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:45.030373 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:45.030739 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.033005 (+ 2266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.035293 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.036646 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.036836 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.037003 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:45.038735 (+ 1732us) server_negotiation.cc:299] Negotiation successful
0618 22:27:45.038920 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12105,"thread_start_us":13648,"threads_started":1}
I20260618 22:27:45.043526 19055 tablet_service.cc:1467] Processing CreateTablet for tablet c86a9596ff6449b3948aa286e181f7a9 (DEFAULT_TABLE table=test-table [id=ccea0b6733f64b2eb863c33439505b94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:45.045534 19055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c86a9596ff6449b3948aa286e181f7a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:45.046660 19288 tablet_bootstrap.cc:492] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7: Bootstrap starting.
I20260618 22:27:45.049744 19289 tablet_bootstrap.cc:492] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f: Bootstrap starting.
I20260618 22:27:45.050009 19288 tablet_bootstrap.cc:654] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:45.051429 19288 log.cc:824] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:45.053579 19289 tablet_bootstrap.cc:654] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:45.055092 19289 log.cc:824] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:45.058795 19292 tablet_bootstrap.cc:492] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76: Bootstrap starting.
I20260618 22:27:45.060858 19288 tablet_bootstrap.cc:492] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7: No bootstrap required, opened a new log
I20260618 22:27:45.061108 19289 tablet_bootstrap.cc:492] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f: No bootstrap required, opened a new log
I20260618 22:27:45.061216 19288 ts_tablet_manager.cc:1397] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.008s
I20260618 22:27:45.061386 19289 ts_tablet_manager.cc:1397] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260618 22:27:45.062033 19292 tablet_bootstrap.cc:654] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:45.063041 19292 log.cc:824] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:45.066354 19292 tablet_bootstrap.cc:492] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76: No bootstrap required, opened a new log
I20260618 22:27:45.066612 19292 ts_tablet_manager.cc:1397] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260618 22:27:45.071697 19288 raft_consensus.cc:348] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } }
I20260618 22:27:45.071697 19289 raft_consensus.cc:348] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } }
I20260618 22:27:45.072170 19289 raft_consensus.cc:374] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:45.072170 19288 raft_consensus.cc:374] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:45.072273 19288 raft_consensus.cc:729] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdde0480a48d43c99b6fd50ca6bd9ce7, State: Initialized, Role: FOLLOWER
I20260618 22:27:45.072279 19289 raft_consensus.cc:729] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba3586b59246465dbef57edb8472d07f, State: Initialized, Role: FOLLOWER
I20260618 22:27:45.072932 19289 consensus_queue.cc:260] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } }
I20260618 22:27:45.072932 19288 consensus_queue.cc:260] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } }
I20260618 22:27:45.074115 19264 heartbeater.cc:503] Master 127.10.242.254:38265 was elected leader, sending a full tablet report...
I20260618 22:27:45.073879 19292 raft_consensus.cc:348] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } }
I20260618 22:27:45.074301 19292 raft_consensus.cc:374] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:45.074398 19292 raft_consensus.cc:729] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd7a2d3199942488cace531d1903c76, State: Initialized, Role: FOLLOWER
I20260618 22:27:45.075002 19292 consensus_queue.cc:260] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } }
I20260618 22:27:45.075577 19288 ts_tablet_manager.cc:1428] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260618 22:27:45.076390 19125 heartbeater.cc:503] Master 127.10.242.254:38265 was elected leader, sending a full tablet report...
I20260618 22:27:45.077239 19289 ts_tablet_manager.cc:1428] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f: Time spent starting tablet: real 0.016s user 0.010s sys 0.003s
I20260618 22:27:45.078348 19292 ts_tablet_manager.cc:1428] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
W20260618 22:27:45.142803 18987 tablet.cc:2369] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:27:45.153180 19126 tablet.cc:2369] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:27:45.172477 19265 tablet.cc:2369] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:27:45.188524 19295 raft_consensus.cc:482] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:27:45.188817 19295 raft_consensus.cc:504] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } }
I20260618 22:27:45.190478 19295 leader_election.cc:288] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdde0480a48d43c99b6fd50ca6bd9ce7 (127.10.242.193:43591), cfd7a2d3199942488cace531d1903c76 (127.10.242.194:37169)
I20260618 22:27:45.196228 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.191004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:32991 (local address 127.10.242.194:37169)
0618 22:27:45.191196 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:45.191203 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:45.191409 (+ 206us) server_negotiation.cc:407] Connection header received
0618 22:27:45.191622 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:45.191627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:45.191686 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:45.191771 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:45.192205 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.192926 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.194575 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.195362 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.195510 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.195584 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:45.195849 (+ 265us) server_negotiation.cc:299] Negotiation successful
0618 22:27:45.195966 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260618 22:27:45.196234 19153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.190781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:37169 (local address 127.10.242.195:32991)
0618 22:27:45.191310 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:45.191329 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:45.191492 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:45.191871 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:45.191876 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:45.191904 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:45.192066 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.192074 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.193066 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.193070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.194412 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.194424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.195510 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.195516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.195698 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.195722 (+ 24us) client_negotiation.cc:769] Sending connection context
0618 22:27:45.195817 (+ 95us) client_negotiation.cc:240] Negotiation successful
0618 22:27:45.195956 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388}
I20260618 22:27:45.196455 19285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.190926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:35541 (local address 127.10.242.193:43591)
0618 22:27:45.191144 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:45.191150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:45.191701 (+ 551us) server_negotiation.cc:407] Connection header received
0618 22:27:45.191865 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:45.191869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:45.191935 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:45.192011 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:45.192540 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.193294 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.194903 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.195646 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.195794 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.195863 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:45.196109 (+ 246us) server_negotiation.cc:299] Negotiation successful
0618 22:27:45.196230 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:27:45.196579 19297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.190918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:43591 (local address 127.10.242.195:35541)
0618 22:27:45.191617 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:45.191630 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:45.191753 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:45.192039 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:45.192044 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:45.192066 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:45.192397 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.192405 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.193436 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.193440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.194757 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.194768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.195840 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.195846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.195977 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.195999 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:27:45.196074 (+ 75us) client_negotiation.cc:240] Negotiation successful
0618 22:27:45.196199 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":363,"threads_started":1}
I20260618 22:27:45.197363 19080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c86a9596ff6449b3948aa286e181f7a9" candidate_uuid: "ba3586b59246465dbef57edb8472d07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd7a2d3199942488cace531d1903c76" is_pre_election: true
I20260618 22:27:45.197531 18936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c86a9596ff6449b3948aa286e181f7a9" candidate_uuid: "ba3586b59246465dbef57edb8472d07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" is_pre_election: true
I20260618 22:27:45.197940 19080 raft_consensus.cc:2454] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba3586b59246465dbef57edb8472d07f in term 0.
I20260618 22:27:45.197969 18936 raft_consensus.cc:2454] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba3586b59246465dbef57edb8472d07f in term 0.
I20260618 22:27:45.198602 19146 leader_election.cc:302] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba3586b59246465dbef57edb8472d07f, bdde0480a48d43c99b6fd50ca6bd9ce7; no voters:
I20260618 22:27:45.198956 19295 raft_consensus.cc:2781] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:27:45.199129 19295 raft_consensus.cc:482] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:27:45.199229 19295 raft_consensus.cc:3037] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:45.201432 19295 raft_consensus.cc:504] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } }
I20260618 22:27:45.201977 19295 leader_election.cc:288] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [CANDIDATE]: Term 1 election: Requested vote from peers bdde0480a48d43c99b6fd50ca6bd9ce7 (127.10.242.193:43591), cfd7a2d3199942488cace531d1903c76 (127.10.242.194:37169)
I20260618 22:27:45.202404 18936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c86a9596ff6449b3948aa286e181f7a9" candidate_uuid: "ba3586b59246465dbef57edb8472d07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7"
I20260618 22:27:45.202461 19080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c86a9596ff6449b3948aa286e181f7a9" candidate_uuid: "ba3586b59246465dbef57edb8472d07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd7a2d3199942488cace531d1903c76"
I20260618 22:27:45.202575 18936 raft_consensus.cc:3037] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:45.202654 19080 raft_consensus.cc:3037] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:45.204710 18936 raft_consensus.cc:2454] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba3586b59246465dbef57edb8472d07f in term 1.
I20260618 22:27:45.204749 19080 raft_consensus.cc:2454] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba3586b59246465dbef57edb8472d07f in term 1.
I20260618 22:27:45.205116 19146 leader_election.cc:302] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba3586b59246465dbef57edb8472d07f, bdde0480a48d43c99b6fd50ca6bd9ce7; no voters:
I20260618 22:27:45.205430 19295 raft_consensus.cc:2781] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:45.205894 19295 raft_consensus.cc:686] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [term 1 LEADER]: Becoming Leader. State: Replica: ba3586b59246465dbef57edb8472d07f, State: Running, Role: LEADER
I20260618 22:27:45.206370 19295 consensus_queue.cc:237] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } }
I20260618 22:27:45.210951 18778 catalog_manager.cc:5495] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f reported cstate change: term changed from 0 to 1, leader changed from <none> to ba3586b59246465dbef57edb8472d07f (127.10.242.195). New cstate: current_term: 1 leader_uuid: "ba3586b59246465dbef57edb8472d07f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } health_report { overall_health: UNKNOWN } } }
I20260618 22:27:45.263938 19284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.259281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37418 (local address 127.10.242.195:36539)
0618 22:27:45.259444 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:45.259448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:45.259464 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:27:45.259546 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:45.259550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:45.259621 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:45.259750 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:27:45.260060 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.260758 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.262020 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.262468 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.262572 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.262928 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:27:45.263496 (+ 568us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:27:45.263599 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:45.263675 (+ 76us) server_negotiation.cc:299] Negotiation successful
0618 22:27:45.263739 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:27:45.272035 18936 raft_consensus.cc:1261] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 FOLLOWER]: Refusing update from remote peer ba3586b59246465dbef57edb8472d07f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:45.272042 19080 raft_consensus.cc:1261] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76 [term 1 FOLLOWER]: Refusing update from remote peer ba3586b59246465dbef57edb8472d07f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:45.273005 19298 consensus_queue.cc:1035] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:45.273442 19295 consensus_queue.cc:1035] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:45.283174 19301 mvcc.cc:205] Tried to move back new op lower bound from 7298341540944269312 to 7298341540688936960. Current Snapshot: MvccSnapshot[applied={T|T < 7298341540944269312}]
I20260618 22:27:45.285785 19302 mvcc.cc:205] Tried to move back new op lower bound from 7298341540944269312 to 7298341540688936960. Current Snapshot: MvccSnapshot[applied={T|T < 7298341540944269312}]
I20260618 22:27:45.288522 19303 mvcc.cc:205] Tried to move back new op lower bound from 7298341540944269312 to 7298341540688936960. Current Snapshot: MvccSnapshot[applied={T|T < 7298341540944269312}]
I20260618 22:27:45.303045 18777 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:45.303385 18777 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:45.308133 18777 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442: Sending DeleteTablet for 3 replicas of tablet c86a9596ff6449b3948aa286e181f7a9
I20260618 22:27:45.309229 18916 tablet_service.cc:1514] Processing DeleteTablet for tablet c86a9596ff6449b3948aa286e181f7a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:45 UTC) from {username='oryx', principal='oryx/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:51838
I20260618 22:27:45.309643 19194 tablet_service.cc:1514] Processing DeleteTablet for tablet c86a9596ff6449b3948aa286e181f7a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:45 UTC) from {username='oryx', principal='oryx/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:37414
I20260618 22:27:45.310246 19193 tablet_service.cc:1514] Processing DeleteTablet for tablet c86a9596ff6449b3948aa286e181f7a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:45 UTC) from {username='oryx', principal='oryx/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:37414
I20260618 22:27:45.312165 19055 tablet_service.cc:1514] Processing DeleteTablet for tablet c86a9596ff6449b3948aa286e181f7a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:45 UTC) from {username='oryx', principal='oryx/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:56342
I20260618 22:27:45.314405 18778 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59038:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:27:45.315040 18778 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:27:45.320180 19312 tablet_replica.cc:330] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f: stopping tablet replica
I20260618 22:27:45.320672 19312 raft_consensus.cc:2229] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:27:45.321381 19312 raft_consensus.cc:2258] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:45.323225 18915 tablet_service.cc:1467] Processing CreateTablet for tablet 935f4ac5c4bd4ba8b684331b6843a604 (DEFAULT_TABLE table=test-table [id=9378a8636fbe45f5839172a895d0b869]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:45.323846 18915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 935f4ac5c4bd4ba8b684331b6843a604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:45.324488 19313 tablet_replica.cc:330] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76: stopping tablet replica
I20260618 22:27:45.325069 19313 raft_consensus.cc:2229] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:45.325582 19313 raft_consensus.cc:2258] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:45.326318 19311 tablet_replica.cc:330] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7: stopping tablet replica
I20260618 22:27:45.326306 19312 ts_tablet_manager.cc:1905] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:45.326700 19311 raft_consensus.cc:2229] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:45.327138 19311 raft_consensus.cc:2258] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:45.328225 19055 tablet_service.cc:1467] Processing CreateTablet for tablet 935f4ac5c4bd4ba8b684331b6843a604 (DEFAULT_TABLE table=test-table [id=9378a8636fbe45f5839172a895d0b869]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:45.328648 19055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 935f4ac5c4bd4ba8b684331b6843a604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:45.329757 19313 ts_tablet_manager.cc:1905] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:45.332298 19194 tablet_service.cc:1467] Processing CreateTablet for tablet 935f4ac5c4bd4ba8b684331b6843a604 (DEFAULT_TABLE table=test-table [id=9378a8636fbe45f5839172a895d0b869]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:45.332738 19194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 935f4ac5c4bd4ba8b684331b6843a604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:45.334285 19312 ts_tablet_manager.cc:1918] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:27:45.334460 19312 log.cc:1196] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/c86a9596ff6449b3948aa286e181f7a9
I20260618 22:27:45.335038 19312 ts_tablet_manager.cc:1939] T c86a9596ff6449b3948aa286e181f7a9 P ba3586b59246465dbef57edb8472d07f: Deleting consensus metadata
I20260618 22:27:45.336234 19313 ts_tablet_manager.cc:1918] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:27:45.336380 19313 log.cc:1196] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/c86a9596ff6449b3948aa286e181f7a9
I20260618 22:27:45.336851 19313 ts_tablet_manager.cc:1939] T c86a9596ff6449b3948aa286e181f7a9 P cfd7a2d3199942488cace531d1903c76: Deleting consensus metadata
I20260618 22:27:45.337970 18766 catalog_manager.cc:4841] TS ba3586b59246465dbef57edb8472d07f (127.10.242.195:36539): tablet c86a9596ff6449b3948aa286e181f7a9 (table test-table [id=ccea0b6733f64b2eb863c33439505b94]) successfully deleted
I20260618 22:27:45.338411 19311 ts_tablet_manager.cc:1905] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 22:27:45.338430 18766 catalog_manager.cc:4808] TS ba3586b59246465dbef57edb8472d07f (127.10.242.195:36539): delete failed for tablet c86a9596ff6449b3948aa286e181f7a9 because the tablet was not found. No further retry: Not found: Tablet not found: c86a9596ff6449b3948aa286e181f7a9
I20260618 22:27:45.338733 18766 catalog_manager.cc:4841] TS cfd7a2d3199942488cace531d1903c76 (127.10.242.194:37169): tablet c86a9596ff6449b3948aa286e181f7a9 (table test-table [id=ccea0b6733f64b2eb863c33439505b94]) successfully deleted
I20260618 22:27:45.341316 19289 tablet_bootstrap.cc:492] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f: Bootstrap starting.
I20260618 22:27:45.342895 19289 tablet_bootstrap.cc:654] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:45.344468 19292 tablet_bootstrap.cc:492] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76: Bootstrap starting.
I20260618 22:27:45.345980 19292 tablet_bootstrap.cc:654] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:45.346967 19288 tablet_bootstrap.cc:492] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7: Bootstrap starting.
I20260618 22:27:45.348672 19288 tablet_bootstrap.cc:654] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:45.350615 19311 ts_tablet_manager.cc:1918] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:27:45.350786 19311 log.cc:1196] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/c86a9596ff6449b3948aa286e181f7a9
I20260618 22:27:45.351332 19311 ts_tablet_manager.cc:1939] T c86a9596ff6449b3948aa286e181f7a9 P bdde0480a48d43c99b6fd50ca6bd9ce7: Deleting consensus metadata
I20260618 22:27:45.353317 19292 tablet_bootstrap.cc:492] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76: No bootstrap required, opened a new log
I20260618 22:27:45.353472 19292 ts_tablet_manager.cc:1397] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260618 22:27:45.353520 18764 catalog_manager.cc:4841] TS bdde0480a48d43c99b6fd50ca6bd9ce7 (127.10.242.193:43591): tablet c86a9596ff6449b3948aa286e181f7a9 (table test-table [id=ccea0b6733f64b2eb863c33439505b94]) successfully deleted
I20260618 22:27:45.354300 19288 tablet_bootstrap.cc:492] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7: No bootstrap required, opened a new log
I20260618 22:27:45.354282 19292 raft_consensus.cc:348] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:45.354436 19288 ts_tablet_manager.cc:1397] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260618 22:27:45.354516 19292 raft_consensus.cc:374] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:45.354971 19292 raft_consensus.cc:729] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd7a2d3199942488cace531d1903c76, State: Initialized, Role: FOLLOWER
I20260618 22:27:45.355420 19289 tablet_bootstrap.cc:492] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f: No bootstrap required, opened a new log
I20260618 22:27:45.355324 19292 consensus_queue.cc:260] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:45.355674 19289 ts_tablet_manager.cc:1397] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.013s
I20260618 22:27:45.355969 19288 raft_consensus.cc:348] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:45.356186 19288 raft_consensus.cc:374] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:45.356278 19288 raft_consensus.cc:729] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdde0480a48d43c99b6fd50ca6bd9ce7, State: Initialized, Role: FOLLOWER
I20260618 22:27:45.356666 19289 raft_consensus.cc:348] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:45.356861 19292 ts_tablet_manager.cc:1428] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 22:27:45.356977 19289 raft_consensus.cc:374] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:45.357097 19289 raft_consensus.cc:729] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba3586b59246465dbef57edb8472d07f, State: Initialized, Role: FOLLOWER
I20260618 22:27:45.357363 19289 consensus_queue.cc:260] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:45.357779 19288 consensus_queue.cc:260] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:45.357944 19289 ts_tablet_manager.cc:1428] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 22:27:45.358414 19288 ts_tablet_manager.cc:1428] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260618 22:27:45.568779 19298 raft_consensus.cc:482] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:27:45.569000 19298 raft_consensus.cc:504] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:45.569679 19298 leader_election.cc:288] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdde0480a48d43c99b6fd50ca6bd9ce7 (127.10.242.193:43591), cfd7a2d3199942488cace531d1903c76 (127.10.242.194:37169)
I20260618 22:27:45.570308 19080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "935f4ac5c4bd4ba8b684331b6843a604" candidate_uuid: "ba3586b59246465dbef57edb8472d07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd7a2d3199942488cace531d1903c76" is_pre_election: true
I20260618 22:27:45.570403 18936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "935f4ac5c4bd4ba8b684331b6843a604" candidate_uuid: "ba3586b59246465dbef57edb8472d07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" is_pre_election: true
I20260618 22:27:45.570572 19080 raft_consensus.cc:2454] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba3586b59246465dbef57edb8472d07f in term 0.
I20260618 22:27:45.570616 18936 raft_consensus.cc:2454] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba3586b59246465dbef57edb8472d07f in term 0.
I20260618 22:27:45.571018 19148 leader_election.cc:302] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba3586b59246465dbef57edb8472d07f, cfd7a2d3199942488cace531d1903c76; no voters:
I20260618 22:27:45.571338 19298 raft_consensus.cc:2781] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:27:45.571465 19298 raft_consensus.cc:482] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:27:45.571576 19298 raft_consensus.cc:3037] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:45.574760 19298 raft_consensus.cc:504] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:45.575512 19298 leader_election.cc:288] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [CANDIDATE]: Term 1 election: Requested vote from peers bdde0480a48d43c99b6fd50ca6bd9ce7 (127.10.242.193:43591), cfd7a2d3199942488cace531d1903c76 (127.10.242.194:37169)
I20260618 22:27:45.576033 18936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "935f4ac5c4bd4ba8b684331b6843a604" candidate_uuid: "ba3586b59246465dbef57edb8472d07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7"
I20260618 22:27:45.576086 19080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "935f4ac5c4bd4ba8b684331b6843a604" candidate_uuid: "ba3586b59246465dbef57edb8472d07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd7a2d3199942488cace531d1903c76"
I20260618 22:27:45.576282 18936 raft_consensus.cc:3037] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:45.576344 19080 raft_consensus.cc:3037] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:45.578496 18936 raft_consensus.cc:2454] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba3586b59246465dbef57edb8472d07f in term 1.
I20260618 22:27:45.578907 19146 leader_election.cc:302] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba3586b59246465dbef57edb8472d07f, bdde0480a48d43c99b6fd50ca6bd9ce7; no voters:
I20260618 22:27:45.579146 19298 raft_consensus.cc:2781] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:45.579301 19298 raft_consensus.cc:686] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [term 1 LEADER]: Becoming Leader. State: Replica: ba3586b59246465dbef57edb8472d07f, State: Running, Role: LEADER
I20260618 22:27:45.579226 19080 raft_consensus.cc:2454] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba3586b59246465dbef57edb8472d07f in term 1.
I20260618 22:27:45.579547 19298 consensus_queue.cc:237] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:45.582201 18778 catalog_manager.cc:5495] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f reported cstate change: term changed from 0 to 1, leader changed from <none> to ba3586b59246465dbef57edb8472d07f (127.10.242.195). New cstate: current_term: 1 leader_uuid: "ba3586b59246465dbef57edb8472d07f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } health_report { overall_health: HEALTHY } } }
Jun 18 22:27:45 dist-test-slave-0956 krb5kdc[18734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821663, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.254@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM
I20260618 22:27:45.633323 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.619438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:38265 (local address 127.0.0.1:59050)
0618 22:27:45.620076 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:45.620100 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:45.620286 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:45.620644 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:45.620648 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:45.621337 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:45.621540 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.621550 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.622578 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.622581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.623538 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.623550 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.624229 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.624234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.624388 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.625124 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:45.625147 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:45.627494 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:45.630583 (+ 3089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:45.630595 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:45.630601 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:45.630997 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:45.631328 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:45.631332 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:45.631334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:45.631448 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:45.631986 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:45.631995 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:45.632195 (+ 200us) client_negotiation.cc:769] Sending connection context
0618 22:27:45.632450 (+ 255us) client_negotiation.cc:240] Negotiation successful
0618 22:27:45.632779 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":341,"threads_started":1}
I20260618 22:27:45.633339 19324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.619525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59050 (local address 127.10.242.254:38265)
0618 22:27:45.620382 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:45.620387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:45.620407 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:27:45.620444 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:45.620447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:45.620504 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:45.620589 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:45.621694 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.622456 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.623678 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.624084 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.624195 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.627686 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:45.627690 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:45.627694 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:45.627720 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:45.630456 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:45.631102 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:45.631108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:45.631112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:45.631235 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:45.631539 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:45.631545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:45.631548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:45.631809 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:45.631941 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:45.632560 (+ 619us) server_negotiation.cc:299] Negotiation successful
0618 22:27:45.632903 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":153,"server-negotiator.queue_time_us":739,"thread_start_us":559,"threads_started":1}
I20260618 22:27:45.638731 18778 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:59050:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260618 22:27:45.640017 18778 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:27:45.647204 18915 tablet_service.cc:1467] Processing CreateTablet for tablet f65f60a3a7ce415087b5e626c6bde031 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bf6535a9c73c41b59f1274bafbdc64f0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:27:45.647387 19194 tablet_service.cc:1467] Processing CreateTablet for tablet f65f60a3a7ce415087b5e626c6bde031 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bf6535a9c73c41b59f1274bafbdc64f0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:27:45.647377 19055 tablet_service.cc:1467] Processing CreateTablet for tablet f65f60a3a7ce415087b5e626c6bde031 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bf6535a9c73c41b59f1274bafbdc64f0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:27:45.647756 18915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f65f60a3a7ce415087b5e626c6bde031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:45.647866 19194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f65f60a3a7ce415087b5e626c6bde031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:45.647904 19055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f65f60a3a7ce415087b5e626c6bde031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:45.655048 19292 tablet_bootstrap.cc:492] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76: Bootstrap starting.
I20260618 22:27:45.655439 19288 tablet_bootstrap.cc:492] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7: Bootstrap starting.
I20260618 22:27:45.655788 19289 tablet_bootstrap.cc:492] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f: Bootstrap starting.
I20260618 22:27:45.656669 19292 tablet_bootstrap.cc:654] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:45.657040 19288 tablet_bootstrap.cc:654] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:45.657482 19289 tablet_bootstrap.cc:654] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:45.667636 19292 tablet_bootstrap.cc:492] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76: No bootstrap required, opened a new log
I20260618 22:27:45.667639 19289 tablet_bootstrap.cc:492] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f: No bootstrap required, opened a new log
I20260618 22:27:45.667817 19289 ts_tablet_manager.cc:1397] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.002s
I20260618 22:27:45.667945 19292 ts_tablet_manager.cc:1397] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.009s
I20260618 22:27:45.668605 19289 raft_consensus.cc:348] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:45.668620 19292 raft_consensus.cc:348] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:45.668864 19289 raft_consensus.cc:374] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:45.668947 19289 raft_consensus.cc:729] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba3586b59246465dbef57edb8472d07f, State: Initialized, Role: FOLLOWER
I20260618 22:27:45.668977 19292 raft_consensus.cc:374] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:45.669271 19292 raft_consensus.cc:729] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd7a2d3199942488cace531d1903c76, State: Initialized, Role: FOLLOWER
I20260618 22:27:45.669632 19292 consensus_queue.cc:260] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:45.670024 19288 tablet_bootstrap.cc:492] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7: No bootstrap required, opened a new log
I20260618 22:27:45.670230 19288 ts_tablet_manager.cc:1397] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260618 22:27:45.670149 19289 consensus_queue.cc:260] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:45.670140 19296 tablet_replica.cc:439] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } }
I20260618 22:27:45.670440 19296 tablet_replica.cc:442] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:27:45.670728 19289 ts_tablet_manager.cc:1428] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 22:27:45.670956 19288 raft_consensus.cc:348] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:45.671195 19288 raft_consensus.cc:374] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:45.671270 19288 raft_consensus.cc:729] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdde0480a48d43c99b6fd50ca6bd9ce7, State: Initialized, Role: FOLLOWER
I20260618 22:27:45.671260 19298 tablet_replica.cc:439] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } }
I20260618 22:27:45.671545 19298 tablet_replica.cc:442] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:27:45.671540 19288 consensus_queue.cc:260] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:45.671970 19294 tablet_replica.cc:439] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } }
I20260618 22:27:45.672329 19294 tablet_replica.cc:442] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:27:45.672762 19288 ts_tablet_manager.cc:1428] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 22:27:45.675792 19292 ts_tablet_manager.cc:1428] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20260618 22:27:45.786165 19294 raft_consensus.cc:482] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:27:45.786367 19294 raft_consensus.cc:504] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:45.787813 19294 leader_election.cc:288] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfd7a2d3199942488cace531d1903c76 (127.10.242.194:37169), ba3586b59246465dbef57edb8472d07f (127.10.242.195:36539)
I20260618 22:27:45.794482 19329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.788135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:59227 (local address 127.10.242.194:37169)
0618 22:27:45.788824 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:45.788830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:45.788851 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:27:45.788931 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:45.788936 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:45.789010 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:45.789198 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:45.789783 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.790742 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.792597 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.793554 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.793694 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.793763 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:45.794069 (+ 306us) server_negotiation.cc:299] Negotiation successful
0618 22:27:45.794206 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":398,"threads_started":1}
I20260618 22:27:45.794459 19330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.788718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:36539 (local address 127.10.242.193:58013)
0618 22:27:45.789314 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:45.789343 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:45.789444 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:45.789771 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:45.789776 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:45.789798 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:45.790023 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.790030 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.791029 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.791033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.792532 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.792549 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.793680 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.793686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.793866 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.793889 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:27:45.793961 (+ 72us) client_negotiation.cc:240] Negotiation successful
0618 22:27:45.794079 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":2,"thread_start_us":304,"threads_started":1}
I20260618 22:27:45.794469 19331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.788860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:58013 (local address 127.10.242.195:36539)
0618 22:27:45.789513 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:45.789519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:45.789537 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:27:45.789596 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:45.789599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:45.789653 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:45.789750 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:27:45.790157 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.790879 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.792677 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.793536 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.793672 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.793762 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:45.794001 (+ 239us) server_negotiation.cc:299] Negotiation successful
0618 22:27:45.794182 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":467,"threads_started":1}
I20260618 22:27:45.794740 19328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.788012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:37169 (local address 127.10.242.193:59227)
0618 22:27:45.788457 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:45.788478 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:45.788641 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:45.789390 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:45.789396 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:45.789427 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:27:45.789646 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.789662 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.790867 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.790872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.792444 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.792459 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.793791 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.793798 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.793923 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.793948 (+ 25us) client_negotiation.cc:769] Sending connection context
0618 22:27:45.794034 (+ 86us) client_negotiation.cc:240] Negotiation successful
0618 22:27:45.794169 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":38,"thread_start_us":239,"threads_started":1}
I20260618 22:27:45.795272 19080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f65f60a3a7ce415087b5e626c6bde031" candidate_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd7a2d3199942488cace531d1903c76" is_pre_election: true
I20260618 22:27:45.795284 19214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f65f60a3a7ce415087b5e626c6bde031" candidate_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba3586b59246465dbef57edb8472d07f" is_pre_election: true
I20260618 22:27:45.795503 19080 raft_consensus.cc:2454] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdde0480a48d43c99b6fd50ca6bd9ce7 in term 0.
I20260618 22:27:45.795593 19214 raft_consensus.cc:2454] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdde0480a48d43c99b6fd50ca6bd9ce7 in term 0.
I20260618 22:27:45.796017 18869 leader_election.cc:302] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdde0480a48d43c99b6fd50ca6bd9ce7, cfd7a2d3199942488cace531d1903c76; no voters:
I20260618 22:27:45.796384 19294 raft_consensus.cc:2781] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:27:45.796512 19294 raft_consensus.cc:482] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:27:45.796599 19294 raft_consensus.cc:3037] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:45.798743 19294 raft_consensus.cc:504] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:45.799299 19294 leader_election.cc:288] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [CANDIDATE]: Term 1 election: Requested vote from peers cfd7a2d3199942488cace531d1903c76 (127.10.242.194:37169), ba3586b59246465dbef57edb8472d07f (127.10.242.195:36539)
I20260618 22:27:45.799803 19080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f65f60a3a7ce415087b5e626c6bde031" candidate_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd7a2d3199942488cace531d1903c76"
I20260618 22:27:45.799909 19214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f65f60a3a7ce415087b5e626c6bde031" candidate_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba3586b59246465dbef57edb8472d07f"
I20260618 22:27:45.800010 19080 raft_consensus.cc:3037] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:45.800112 19214 raft_consensus.cc:3037] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:45.801975 19080 raft_consensus.cc:2454] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdde0480a48d43c99b6fd50ca6bd9ce7 in term 1.
I20260618 22:27:45.802335 19214 raft_consensus.cc:2454] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdde0480a48d43c99b6fd50ca6bd9ce7 in term 1.
I20260618 22:27:45.802413 18869 leader_election.cc:302] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdde0480a48d43c99b6fd50ca6bd9ce7, cfd7a2d3199942488cace531d1903c76; no voters:
I20260618 22:27:45.802734 19294 raft_consensus.cc:2781] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:45.803120 19294 raft_consensus.cc:686] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 LEADER]: Becoming Leader. State: Replica: bdde0480a48d43c99b6fd50ca6bd9ce7, State: Running, Role: LEADER
I20260618 22:27:45.803465 19294 consensus_queue.cc:237] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:45.805270 19332 tablet_replica.cc:439] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7: TxnStatusTablet state changed. Reason: New leader bdde0480a48d43c99b6fd50ca6bd9ce7. Latest consensus state: current_term: 1 leader_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } }
I20260618 22:27:45.805682 19332 tablet_replica.cc:442] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:27:45.806344 19334 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260618 22:27:45.806542 19334 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260618 22:27:45.806816 18778 catalog_manager.cc:5495] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdde0480a48d43c99b6fd50ca6bd9ce7 (127.10.242.193). New cstate: current_term: 1 leader_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } health_report { overall_health: UNKNOWN } } }
I20260618 22:27:45.867794 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.861572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:43591 (local address 127.0.0.1:37444)
0618 22:27:45.861909 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:45.861929 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:45.862076 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:45.862551 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:45.862555 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:45.862564 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:27:45.862706 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.862712 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.864137 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.864141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.865184 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.865196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.866111 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.866115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.866257 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.866311 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:27:45.867337 (+ 1026us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:27:45.867359 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:27:45.867420 (+ 61us) client_negotiation.cc:240] Negotiation successful
0618 22:27:45.867528 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260618 22:27:45.867857 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.861521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37444 (local address 127.10.242.193:43591)
0618 22:27:45.862136 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:45.862142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:45.862162 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:27:45.862258 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:45.862263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:45.862361 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:45.862519 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:27:45.862870 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.863997 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.865314 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.865981 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.866100 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.866513 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:27:45.867235 (+ 722us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:27:45.867367 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:45.867445 (+ 78us) server_negotiation.cc:299] Negotiation successful
0618 22:27:45.867525 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":353,"threads_started":1}
I20260618 22:27:45.871976 19080 raft_consensus.cc:1261] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76 [term 1 FOLLOWER]: Refusing update from remote peer bdde0480a48d43c99b6fd50ca6bd9ce7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:45.872181 19214 raft_consensus.cc:1261] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f [term 1 FOLLOWER]: Refusing update from remote peer bdde0480a48d43c99b6fd50ca6bd9ce7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:45.872713 19294 consensus_queue.cc:1035] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:45.873034 19332 consensus_queue.cc:1035] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:45.878633 19298 tablet_replica.cc:439] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f: TxnStatusTablet state changed. Reason: New leader bdde0480a48d43c99b6fd50ca6bd9ce7. Latest consensus state: current_term: 1 leader_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } }
I20260618 22:27:45.878925 19298 tablet_replica.cc:442] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:27:45.883510 19298 tablet_replica.cc:439] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } }
I20260618 22:27:45.883814 19298 tablet_replica.cc:442] T f65f60a3a7ce415087b5e626c6bde031 P ba3586b59246465dbef57edb8472d07f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:27:45.886483 19294 tablet_replica.cc:439] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } }
I20260618 22:27:45.886897 19294 tablet_replica.cc:442] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:27:45.889271 19296 tablet_replica.cc:439] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76: TxnStatusTablet state changed. Reason: New leader bdde0480a48d43c99b6fd50ca6bd9ce7. Latest consensus state: current_term: 1 leader_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } }
I20260618 22:27:45.889577 19296 tablet_replica.cc:442] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:27:45.891084 19294 tablet_replica.cc:439] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } }
I20260618 22:27:45.891393 19294 tablet_replica.cc:442] T f65f60a3a7ce415087b5e626c6bde031 P bdde0480a48d43c99b6fd50ca6bd9ce7: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:27:45.903772 19296 tablet_replica.cc:439] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } } }
I20260618 22:27:45.904080 19296 tablet_replica.cc:442] T f65f60a3a7ce415087b5e626c6bde031 P cfd7a2d3199942488cace531d1903c76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:27:45.941449 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.935784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:40735 (local address 127.10.242.193:43591)
0618 22:27:45.935979 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:45.935986 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:45.936615 (+ 629us) server_negotiation.cc:407] Connection header received
0618 22:27:45.936811 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:45.936815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:45.936878 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:45.936950 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:27:45.937487 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.938262 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.939735 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.940160 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.940276 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.940718 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:27:45.940829 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:27:45.940909 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:45.941038 (+ 129us) server_negotiation.cc:299] Negotiation successful
0618 22:27:45.941149 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260618 22:27:45.941782 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.935778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:43591 (local address 127.10.242.195:40735)
0618 22:27:45.936507 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:45.936523 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:45.936679 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:45.936984 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:45.936988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:45.937001 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:27:45.937338 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.937347 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.938397 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.938403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.939589 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.939601 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.940353 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.940359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.940480 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.940538 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:27:45.940921 (+ 383us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:27:45.940940 (+ 19us) client_negotiation.cc:769] Sending connection context
0618 22:27:45.941006 (+ 66us) client_negotiation.cc:240] Negotiation successful
0618 22:27:45.941179 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":381,"threads_started":1}
I20260618 22:27:45.951229 19080 raft_consensus.cc:1261] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76 [term 1 FOLLOWER]: Refusing update from remote peer ba3586b59246465dbef57edb8472d07f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:45.951264 18936 raft_consensus.cc:1261] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 FOLLOWER]: Refusing update from remote peer ba3586b59246465dbef57edb8472d07f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:45.951900 19308 consensus_queue.cc:1035] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:45.952327 19298 consensus_queue.cc:1035] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:45.994102 19331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.987485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:59025 (local address 127.10.242.195:36539)
0618 22:27:45.987651 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:45.987658 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:45.988107 (+ 449us) server_negotiation.cc:407] Connection header received
0618 22:27:45.988356 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:45.988361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:45.988428 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:45.988494 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:27:45.989122 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.990118 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.991625 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.992094 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.992203 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.993158 (+ 955us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:27:45.993286 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:27:45.993386 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:45.993771 (+ 385us) server_negotiation.cc:299] Negotiation successful
0618 22:27:45.993864 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:27:45.994226 19353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:45.987410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:36539 (local address 127.10.242.193:59025)
0618 22:27:45.988019 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:45.988043 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:45.988224 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:45.988528 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:45.988533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:45.988543 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:27:45.988926 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.988939 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.990243 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.990249 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.991507 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:45.991520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:45.992280 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:45.992287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:45.992451 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:45.992953 (+ 502us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:27:45.993400 (+ 447us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:27:45.993413 (+ 13us) client_negotiation.cc:769] Sending connection context
0618 22:27:45.993619 (+ 206us) client_negotiation.cc:240] Negotiation successful
0618 22:27:45.993786 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":279,"threads_started":1}
I20260618 22:27:46.031803 18777 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:46.032076 18777 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:46.035321 18777 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442: Sending DeleteTablet for 3 replicas of tablet 935f4ac5c4bd4ba8b684331b6843a604
I20260618 22:27:46.036111 18915 tablet_service.cc:1514] Processing DeleteTablet for tablet 935f4ac5c4bd4ba8b684331b6843a604 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:46 UTC) from {username='oryx', principal='oryx/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:51838
I20260618 22:27:46.036319 19055 tablet_service.cc:1514] Processing DeleteTablet for tablet 935f4ac5c4bd4ba8b684331b6843a604 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:46 UTC) from {username='oryx', principal='oryx/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:56342
I20260618 22:27:46.036341 19194 tablet_service.cc:1514] Processing DeleteTablet for tablet 935f4ac5c4bd4ba8b684331b6843a604 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:46 UTC) from {username='oryx', principal='oryx/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:37414
I20260618 22:27:46.047195 19356 tablet_replica.cc:330] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7: stopping tablet replica
I20260618 22:27:46.047488 19356 raft_consensus.cc:2229] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:46.047850 19356 raft_consensus.cc:2258] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:46.049265 19356 ts_tablet_manager.cc:1905] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:46.058116 19356 ts_tablet_manager.cc:1918] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:27:46.058292 19356 log.cc:1196] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/935f4ac5c4bd4ba8b684331b6843a604
I20260618 22:27:46.058779 19356 ts_tablet_manager.cc:1939] T 935f4ac5c4bd4ba8b684331b6843a604 P bdde0480a48d43c99b6fd50ca6bd9ce7: Deleting consensus metadata
I20260618 22:27:46.059856 19358 tablet_replica.cc:330] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f: stopping tablet replica
I20260618 22:27:46.060060 18764 catalog_manager.cc:4841] TS bdde0480a48d43c99b6fd50ca6bd9ce7 (127.10.242.193:43591): tablet 935f4ac5c4bd4ba8b684331b6843a604 (table test-table [id=9378a8636fbe45f5839172a895d0b869]) successfully deleted
I20260618 22:27:46.060262 19358 raft_consensus.cc:2229] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:27:46.060756 19358 raft_consensus.cc:2258] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:46.061905 19358 ts_tablet_manager.cc:1905] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:46.069185 19358 ts_tablet_manager.cc:1918] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:27:46.069829 19357 tablet_replica.cc:330] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76: stopping tablet replica
I20260618 22:27:46.070065 19357 raft_consensus.cc:2229] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:46.070355 19357 raft_consensus.cc:2258] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:46.070616 19358 log.cc:1196] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/935f4ac5c4bd4ba8b684331b6843a604
I20260618 22:27:46.071067 19358 ts_tablet_manager.cc:1939] T 935f4ac5c4bd4ba8b684331b6843a604 P ba3586b59246465dbef57edb8472d07f: Deleting consensus metadata
I20260618 22:27:46.072201 18766 catalog_manager.cc:4841] TS ba3586b59246465dbef57edb8472d07f (127.10.242.195:36539): tablet 935f4ac5c4bd4ba8b684331b6843a604 (table test-table [id=9378a8636fbe45f5839172a895d0b869]) successfully deleted
I20260618 22:27:46.072635 19357 ts_tablet_manager.cc:1905] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:46.079752 19357 ts_tablet_manager.cc:1918] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:27:46.079897 19357 log.cc:1196] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/935f4ac5c4bd4ba8b684331b6843a604
I20260618 22:27:46.080304 19357 ts_tablet_manager.cc:1939] T 935f4ac5c4bd4ba8b684331b6843a604 P cfd7a2d3199942488cace531d1903c76: Deleting consensus metadata
I20260618 22:27:46.081815 18766 catalog_manager.cc:4841] TS cfd7a2d3199942488cace531d1903c76 (127.10.242.194:37169): tablet 935f4ac5c4bd4ba8b684331b6843a604 (table test-table [id=9378a8636fbe45f5839172a895d0b869]) successfully deleted
Jun 18 22:27:46 dist-test-slave-0956 krb5kdc[18734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 18 22:27:46 dist-test-slave-0956 krb5kdc[18734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:46 dist-test-slave-0956 krb5kdc[18734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
W20260618 22:27:46.111316 19324 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:46.096815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46130 (local address 127.10.242.254:38265)
0618 22:27:46.096992 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:46.096999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:46.097484 (+ 485us) server_negotiation.cc:407] Connection header received
0618 22:27:46.097673 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:46.097679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:46.097756 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:46.097840 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:46.099111 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:46.099835 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:46.101037 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:46.101534 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:46.101646 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:46.111040 (+ 9394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46130: BlockingRecv error: recv got EOF from 127.0.0.1:46130 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 18 22:27:46 dist-test-slave-0956 krb5kdc[18734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM
I20260618 22:27:46.129361 19324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:46.114466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46146 (local address 127.10.242.254:38265)
0618 22:27:46.114645 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:46.114651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:46.114937 (+ 286us) server_negotiation.cc:407] Connection header received
0618 22:27:46.115087 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:46.115092 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:46.115170 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:46.115282 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:46.116381 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:46.117532 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:46.118720 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:46.119326 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:46.119458 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:46.123276 (+ 3818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:46.123283 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:46.123288 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:46.123360 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:46.126519 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:46.127150 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:46.127156 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:46.127159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:46.127329 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:46.127767 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:46.127773 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:46.127777 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:46.128139 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:46.128441 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:46.128722 (+ 281us) server_negotiation.cc:299] Negotiation successful
0618 22:27:46.128956 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260618 22:27:46.133373 18777 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:27:46.134117 18777 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:27:46.141374 18915 tablet_service.cc:1467] Processing CreateTablet for tablet 1f104ea9315f463086ab2e16d7d272a7 (DEFAULT_TABLE table=test-table [id=3b666dfef59541049309c465f41ee1fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:46.141732 19055 tablet_service.cc:1467] Processing CreateTablet for tablet 1f104ea9315f463086ab2e16d7d272a7 (DEFAULT_TABLE table=test-table [id=3b666dfef59541049309c465f41ee1fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:46.141911 18915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f104ea9315f463086ab2e16d7d272a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:46.142138 19055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f104ea9315f463086ab2e16d7d272a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:46.142189 19194 tablet_service.cc:1467] Processing CreateTablet for tablet 1f104ea9315f463086ab2e16d7d272a7 (DEFAULT_TABLE table=test-table [id=3b666dfef59541049309c465f41ee1fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:46.142570 19194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f104ea9315f463086ab2e16d7d272a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:46.148476 19292 tablet_bootstrap.cc:492] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76: Bootstrap starting.
I20260618 22:27:46.148634 19288 tablet_bootstrap.cc:492] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7: Bootstrap starting.
I20260618 22:27:46.150161 19292 tablet_bootstrap.cc:654] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:46.150300 19288 tablet_bootstrap.cc:654] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:46.151235 19289 tablet_bootstrap.cc:492] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f: Bootstrap starting.
I20260618 22:27:46.152400 19288 tablet_bootstrap.cc:492] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7: No bootstrap required, opened a new log
I20260618 22:27:46.152534 19288 ts_tablet_manager.cc:1397] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260618 22:27:46.152822 19289 tablet_bootstrap.cc:654] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:46.153332 19288 raft_consensus.cc:348] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:46.153549 19288 raft_consensus.cc:374] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:46.153640 19288 raft_consensus.cc:729] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdde0480a48d43c99b6fd50ca6bd9ce7, State: Initialized, Role: FOLLOWER
I20260618 22:27:46.153916 19288 consensus_queue.cc:260] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:46.155063 19288 ts_tablet_manager.cc:1428] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:27:46.155247 19289 tablet_bootstrap.cc:492] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f: No bootstrap required, opened a new log
I20260618 22:27:46.155397 19289 ts_tablet_manager.cc:1397] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260618 22:27:46.156476 19289 raft_consensus.cc:348] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:46.156723 19289 raft_consensus.cc:374] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:46.156822 19289 raft_consensus.cc:729] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba3586b59246465dbef57edb8472d07f, State: Initialized, Role: FOLLOWER
I20260618 22:27:46.157049 19289 consensus_queue.cc:260] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:46.157898 19289 ts_tablet_manager.cc:1428] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260618 22:27:46.162750 19292 tablet_bootstrap.cc:492] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76: No bootstrap required, opened a new log
I20260618 22:27:46.162922 19292 ts_tablet_manager.cc:1397] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.004s
I20260618 22:27:46.163704 19292 raft_consensus.cc:348] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:46.163976 19292 raft_consensus.cc:374] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:46.164079 19292 raft_consensus.cc:729] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd7a2d3199942488cace531d1903c76, State: Initialized, Role: FOLLOWER
I20260618 22:27:46.164312 19292 consensus_queue.cc:260] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:46.164870 19292 ts_tablet_manager.cc:1428] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 22:27:46.423328 19345 raft_consensus.cc:482] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:27:46.423514 19345 raft_consensus.cc:504] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:46.424147 19345 leader_election.cc:288] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfd7a2d3199942488cace531d1903c76 (127.10.242.194:37169), ba3586b59246465dbef57edb8472d07f (127.10.242.195:36539)
I20260618 22:27:46.424816 19213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f104ea9315f463086ab2e16d7d272a7" candidate_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba3586b59246465dbef57edb8472d07f" is_pre_election: true
I20260618 22:27:46.424880 19079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f104ea9315f463086ab2e16d7d272a7" candidate_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd7a2d3199942488cace531d1903c76" is_pre_election: true
I20260618 22:27:46.425104 19213 raft_consensus.cc:2454] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdde0480a48d43c99b6fd50ca6bd9ce7 in term 0.
I20260618 22:27:46.425132 19079 raft_consensus.cc:2454] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdde0480a48d43c99b6fd50ca6bd9ce7 in term 0.
I20260618 22:27:46.425515 18869 leader_election.cc:302] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba3586b59246465dbef57edb8472d07f, bdde0480a48d43c99b6fd50ca6bd9ce7; no voters:
I20260618 22:27:46.425800 19345 raft_consensus.cc:2781] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:27:46.425906 19345 raft_consensus.cc:482] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:27:46.425992 19345 raft_consensus.cc:3037] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:46.427887 19345 raft_consensus.cc:504] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:46.428390 19345 leader_election.cc:288] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [CANDIDATE]: Term 1 election: Requested vote from peers cfd7a2d3199942488cace531d1903c76 (127.10.242.194:37169), ba3586b59246465dbef57edb8472d07f (127.10.242.195:36539)
I20260618 22:27:46.428828 19079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f104ea9315f463086ab2e16d7d272a7" candidate_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd7a2d3199942488cace531d1903c76"
I20260618 22:27:46.428898 19213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f104ea9315f463086ab2e16d7d272a7" candidate_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba3586b59246465dbef57edb8472d07f"
I20260618 22:27:46.428990 19079 raft_consensus.cc:3037] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:46.429121 19213 raft_consensus.cc:3037] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:46.430917 19079 raft_consensus.cc:2454] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdde0480a48d43c99b6fd50ca6bd9ce7 in term 1.
I20260618 22:27:46.431111 19213 raft_consensus.cc:2454] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdde0480a48d43c99b6fd50ca6bd9ce7 in term 1.
I20260618 22:27:46.431298 18869 leader_election.cc:302] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdde0480a48d43c99b6fd50ca6bd9ce7, cfd7a2d3199942488cace531d1903c76; no voters:
I20260618 22:27:46.431553 19345 raft_consensus.cc:2781] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:46.431699 19345 raft_consensus.cc:686] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 LEADER]: Becoming Leader. State: Replica: bdde0480a48d43c99b6fd50ca6bd9ce7, State: Running, Role: LEADER
I20260618 22:27:46.431964 19345 consensus_queue.cc:237] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:46.434384 18778 catalog_manager.cc:5495] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdde0480a48d43c99b6fd50ca6bd9ce7 (127.10.242.193). New cstate: current_term: 1 leader_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } health_report { overall_health: UNKNOWN } } }
I20260618 22:27:46.465520 19376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:46.460985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37454 (local address 127.10.242.193:43591)
0618 22:27:46.461458 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:46.461463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:46.461478 (+ 15us) server_negotiation.cc:407] Connection header received
0618 22:27:46.461541 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:46.461545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:46.461605 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:46.461682 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:27:46.462096 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:46.462896 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:46.464133 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:46.464518 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:46.464612 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:46.464939 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:27:46.465036 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:27:46.465141 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:46.465228 (+ 87us) server_negotiation.cc:299] Negotiation successful
0618 22:27:46.465290 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":302,"threads_started":1}
I20260618 22:27:46.468369 19213 raft_consensus.cc:1261] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f [term 1 FOLLOWER]: Refusing update from remote peer bdde0480a48d43c99b6fd50ca6bd9ce7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:46.468384 19079 raft_consensus.cc:1261] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76 [term 1 FOLLOWER]: Refusing update from remote peer bdde0480a48d43c99b6fd50ca6bd9ce7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:46.468963 19345 consensus_queue.cc:1035] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:46.469295 19332 consensus_queue.cc:1035] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:46.481989 18777 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:46.482231 18777 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:46.485037 18777 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442: Sending DeleteTablet for 3 replicas of tablet 1f104ea9315f463086ab2e16d7d272a7
I20260618 22:27:46.485824 18915 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f104ea9315f463086ab2e16d7d272a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:46 UTC) from {username='oryx', principal='oryx/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:51838
I20260618 22:27:46.486095 19356 tablet_replica.cc:330] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7: stopping tablet replica
I20260618 22:27:46.486153 19055 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f104ea9315f463086ab2e16d7d272a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:46 UTC) from {username='oryx', principal='oryx/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:56342
I20260618 22:27:46.486303 19356 raft_consensus.cc:2229] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:27:46.486449 19194 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f104ea9315f463086ab2e16d7d272a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:46 UTC) from {username='oryx', principal='oryx/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:37414
I20260618 22:27:46.486697 19357 tablet_replica.cc:330] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76: stopping tablet replica
I20260618 22:27:46.486804 19356 raft_consensus.cc:2258] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:46.486881 19357 raft_consensus.cc:2229] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:46.487074 19357 raft_consensus.cc:2258] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:46.487041 18777 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 22:27:46.487323 19358 tablet_replica.cc:330] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f: stopping tablet replica
I20260618 22:27:46.487490 19358 raft_consensus.cc:2229] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:46.487694 19358 raft_consensus.cc:2258] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:46.487789 19356 ts_tablet_manager.cc:1905] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 22:27:46.487797 18777 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:27:46.489172 19357 ts_tablet_manager.cc:1905] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:46.489557 19358 ts_tablet_manager.cc:1905] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:46.494616 19356 ts_tablet_manager.cc:1918] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:27:46.494779 19356 log.cc:1196] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/1f104ea9315f463086ab2e16d7d272a7
I20260618 22:27:46.495183 19356 ts_tablet_manager.cc:1939] T 1f104ea9315f463086ab2e16d7d272a7 P bdde0480a48d43c99b6fd50ca6bd9ce7: Deleting consensus metadata
I20260618 22:27:46.496186 19358 ts_tablet_manager.cc:1918] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:27:46.496248 18764 catalog_manager.cc:4841] TS bdde0480a48d43c99b6fd50ca6bd9ce7 (127.10.242.193:43591): tablet 1f104ea9315f463086ab2e16d7d272a7 (table test-table [id=3b666dfef59541049309c465f41ee1fd]) successfully deleted
I20260618 22:27:46.497036 19358 log.cc:1196] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/1f104ea9315f463086ab2e16d7d272a7
I20260618 22:27:46.497356 19357 ts_tablet_manager.cc:1918] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:27:46.497498 19357 log.cc:1196] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/1f104ea9315f463086ab2e16d7d272a7
I20260618 22:27:46.497617 19358 ts_tablet_manager.cc:1939] T 1f104ea9315f463086ab2e16d7d272a7 P ba3586b59246465dbef57edb8472d07f: Deleting consensus metadata
I20260618 22:27:46.497890 19357 ts_tablet_manager.cc:1939] T 1f104ea9315f463086ab2e16d7d272a7 P cfd7a2d3199942488cace531d1903c76: Deleting consensus metadata
I20260618 22:27:46.498723 18766 catalog_manager.cc:4841] TS ba3586b59246465dbef57edb8472d07f (127.10.242.195:36539): tablet 1f104ea9315f463086ab2e16d7d272a7 (table test-table [id=3b666dfef59541049309c465f41ee1fd]) successfully deleted
I20260618 22:27:46.498987 18766 catalog_manager.cc:4841] TS cfd7a2d3199942488cace531d1903c76 (127.10.242.194:37169): tablet 1f104ea9315f463086ab2e16d7d272a7 (table test-table [id=3b666dfef59541049309c465f41ee1fd]) successfully deleted
I20260618 22:27:46.500348 18915 tablet_service.cc:1467] Processing CreateTablet for tablet d6664554cdae48f1b66ec1128c8d0f44 (DEFAULT_TABLE table=test-table [id=ff3c16bc01b84da3a6d7792f7d06eb90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:46.500758 18915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6664554cdae48f1b66ec1128c8d0f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:46.500703 19055 tablet_service.cc:1467] Processing CreateTablet for tablet d6664554cdae48f1b66ec1128c8d0f44 (DEFAULT_TABLE table=test-table [id=ff3c16bc01b84da3a6d7792f7d06eb90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:46.501075 19055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6664554cdae48f1b66ec1128c8d0f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:46.501726 19194 tablet_service.cc:1467] Processing CreateTablet for tablet d6664554cdae48f1b66ec1128c8d0f44 (DEFAULT_TABLE table=test-table [id=ff3c16bc01b84da3a6d7792f7d06eb90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:27:46.502077 19194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6664554cdae48f1b66ec1128c8d0f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:46.506884 19292 tablet_bootstrap.cc:492] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76: Bootstrap starting.
I20260618 22:27:46.507038 19288 tablet_bootstrap.cc:492] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7: Bootstrap starting.
I20260618 22:27:46.508342 19292 tablet_bootstrap.cc:654] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:46.508875 19288 tablet_bootstrap.cc:654] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:46.509408 19289 tablet_bootstrap.cc:492] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f: Bootstrap starting.
I20260618 22:27:46.510948 19289 tablet_bootstrap.cc:654] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:46.519337 19292 tablet_bootstrap.cc:492] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76: No bootstrap required, opened a new log
I20260618 22:27:46.519523 19292 ts_tablet_manager.cc:1397] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.005s
I20260618 22:27:46.520099 19289 tablet_bootstrap.cc:492] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f: No bootstrap required, opened a new log
I20260618 22:27:46.520293 19289 ts_tablet_manager.cc:1397] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.008s
I20260618 22:27:46.520334 19292 raft_consensus.cc:348] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:46.520577 19292 raft_consensus.cc:374] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:46.520653 19292 raft_consensus.cc:729] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd7a2d3199942488cace531d1903c76, State: Initialized, Role: FOLLOWER
I20260618 22:27:46.521004 19288 tablet_bootstrap.cc:492] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7: No bootstrap required, opened a new log
I20260618 22:27:46.520908 19292 consensus_queue.cc:260] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:46.521229 19288 ts_tablet_manager.cc:1397] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.006s
I20260618 22:27:46.521359 19289 raft_consensus.cc:348] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:46.521651 19289 raft_consensus.cc:374] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:46.521770 19289 raft_consensus.cc:729] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba3586b59246465dbef57edb8472d07f, State: Initialized, Role: FOLLOWER
I20260618 22:27:46.522027 19289 consensus_queue.cc:260] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:46.521997 19288 raft_consensus.cc:348] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:46.522251 19288 raft_consensus.cc:374] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:46.522337 19288 raft_consensus.cc:729] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdde0480a48d43c99b6fd50ca6bd9ce7, State: Initialized, Role: FOLLOWER
I20260618 22:27:46.522603 19288 consensus_queue.cc:260] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:46.522729 19289 ts_tablet_manager.cc:1428] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 22:27:46.524205 19288 ts_tablet_manager.cc:1428] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 22:27:46.524528 19292 ts_tablet_manager.cc:1428] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260618 22:27:46.616889 19355 raft_consensus.cc:482] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:27:46.617164 19355 raft_consensus.cc:504] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:46.617944 19355 leader_election.cc:288] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdde0480a48d43c99b6fd50ca6bd9ce7 (127.10.242.193:43591), cfd7a2d3199942488cace531d1903c76 (127.10.242.194:37169)
I20260618 22:27:46.618541 19079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6664554cdae48f1b66ec1128c8d0f44" candidate_uuid: "ba3586b59246465dbef57edb8472d07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd7a2d3199942488cace531d1903c76" is_pre_election: true
I20260618 22:27:46.618530 18936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6664554cdae48f1b66ec1128c8d0f44" candidate_uuid: "ba3586b59246465dbef57edb8472d07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" is_pre_election: true
I20260618 22:27:46.618798 18936 raft_consensus.cc:2454] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba3586b59246465dbef57edb8472d07f in term 0.
I20260618 22:27:46.618791 19079 raft_consensus.cc:2454] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba3586b59246465dbef57edb8472d07f in term 0.
I20260618 22:27:46.619230 19148 leader_election.cc:302] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba3586b59246465dbef57edb8472d07f, cfd7a2d3199942488cace531d1903c76; no voters:
I20260618 22:27:46.619530 19355 raft_consensus.cc:2781] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:27:46.619652 19355 raft_consensus.cc:482] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:27:46.619753 19355 raft_consensus.cc:3037] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:46.621891 19355 raft_consensus.cc:504] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:46.622526 19355 leader_election.cc:288] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [CANDIDATE]: Term 1 election: Requested vote from peers bdde0480a48d43c99b6fd50ca6bd9ce7 (127.10.242.193:43591), cfd7a2d3199942488cace531d1903c76 (127.10.242.194:37169)
I20260618 22:27:46.623020 18936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6664554cdae48f1b66ec1128c8d0f44" candidate_uuid: "ba3586b59246465dbef57edb8472d07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7"
I20260618 22:27:46.623100 19079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6664554cdae48f1b66ec1128c8d0f44" candidate_uuid: "ba3586b59246465dbef57edb8472d07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd7a2d3199942488cace531d1903c76"
I20260618 22:27:46.623306 19079 raft_consensus.cc:3037] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:46.623312 18936 raft_consensus.cc:3037] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:46.626325 19079 raft_consensus.cc:2454] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba3586b59246465dbef57edb8472d07f in term 1.
I20260618 22:27:46.626348 18936 raft_consensus.cc:2454] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba3586b59246465dbef57edb8472d07f in term 1.
I20260618 22:27:46.626900 19148 leader_election.cc:302] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: ba3586b59246465dbef57edb8472d07f, bdde0480a48d43c99b6fd50ca6bd9ce7, cfd7a2d3199942488cace531d1903c76; no voters:
I20260618 22:27:46.627205 19355 raft_consensus.cc:2781] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:46.627626 19355 raft_consensus.cc:686] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [term 1 LEADER]: Becoming Leader. State: Replica: ba3586b59246465dbef57edb8472d07f, State: Running, Role: LEADER
I20260618 22:27:46.627919 19355 consensus_queue.cc:237] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } }
I20260618 22:27:46.630786 18779 catalog_manager.cc:5495] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f reported cstate change: term changed from 0 to 1, leader changed from <none> to ba3586b59246465dbef57edb8472d07f (127.10.242.195). New cstate: current_term: 1 leader_uuid: "ba3586b59246465dbef57edb8472d07f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba3586b59246465dbef57edb8472d07f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36539 } health_report { overall_health: HEALTHY } } }
I20260618 22:27:46.654434 19386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:46.649356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41570 (local address 127.10.242.195:36539)
0618 22:27:46.649844 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:46.649848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:46.649860 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:27:46.649904 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:46.649907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:46.649961 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:46.650039 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:27:46.650425 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:46.651151 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:46.652432 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:46.653133 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:46.653259 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:46.653704 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:27:46.653848 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:27:46.653935 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:46.654028 (+ 93us) server_negotiation.cc:299] Negotiation successful
0618 22:27:46.654101 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":225,"threads_started":1}
I20260618 22:27:46.663219 19079 raft_consensus.cc:1261] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76 [term 1 FOLLOWER]: Refusing update from remote peer ba3586b59246465dbef57edb8472d07f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:46.663321 18936 raft_consensus.cc:1261] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 FOLLOWER]: Refusing update from remote peer ba3586b59246465dbef57edb8472d07f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:27:46.663738 19384 consensus_queue.cc:1035] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfd7a2d3199942488cace531d1903c76" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:46.664052 19355 consensus_queue.cc:1035] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdde0480a48d43c99b6fd50ca6bd9ce7" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:27:46.722507 18777 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:46.722816 18777 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:27:46.725903 18777 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b73fef7fbc143a7b5f98f2ad8c99442: Sending DeleteTablet for 3 replicas of tablet d6664554cdae48f1b66ec1128c8d0f44
I20260618 22:27:46.726811 19194 tablet_service.cc:1514] Processing DeleteTablet for tablet d6664554cdae48f1b66ec1128c8d0f44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:46 UTC) from {username='oryx', principal='oryx/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:37414
I20260618 22:27:46.726996 19055 tablet_service.cc:1514] Processing DeleteTablet for tablet d6664554cdae48f1b66ec1128c8d0f44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:46 UTC) from {username='oryx', principal='oryx/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:56342
I20260618 22:27:46.727097 19358 tablet_replica.cc:330] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f: stopping tablet replica
I20260618 22:27:46.727288 19357 tablet_replica.cc:330] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76: stopping tablet replica
I20260618 22:27:46.727275 19358 raft_consensus.cc:2229] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:27:46.727444 19357 raft_consensus.cc:2229] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:46.727586 19358 raft_consensus.cc:2258] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:46.727618 19357 raft_consensus.cc:2258] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:46.728534 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 18850
I20260618 22:27:46.728772 19358 ts_tablet_manager.cc:1905] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:46.730268 18915 tablet_service.cc:1514] Processing DeleteTablet for tablet d6664554cdae48f1b66ec1128c8d0f44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:27:46 UTC) from {username='oryx', principal='oryx/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:51838
I20260618 22:27:46.730561 19356 tablet_replica.cc:330] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7: stopping tablet replica
I20260618 22:27:46.730749 19356 raft_consensus.cc:2229] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:27:46.730960 19356 raft_consensus.cc:2258] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:27:46.732098 19357 ts_tablet_manager.cc:1905] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:46.734665 19356 ts_tablet_manager.cc:1905] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:27:46.735821 19358 ts_tablet_manager.cc:1918] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:27:46.735939 19358 log.cc:1196] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/d6664554cdae48f1b66ec1128c8d0f44
I20260618 22:27:46.736310 19358 ts_tablet_manager.cc:1939] T d6664554cdae48f1b66ec1128c8d0f44 P ba3586b59246465dbef57edb8472d07f: Deleting consensus metadata
I20260618 22:27:46.737404 18766 catalog_manager.cc:4841] TS ba3586b59246465dbef57edb8472d07f (127.10.242.195:36539): tablet d6664554cdae48f1b66ec1128c8d0f44 (table test-table [id=ff3c16bc01b84da3a6d7792f7d06eb90]) successfully deleted
I20260618 22:27:46.738889 19357 ts_tablet_manager.cc:1918] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:27:46.739009 19357 log.cc:1196] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/d6664554cdae48f1b66ec1128c8d0f44
I20260618 22:27:46.739396 19357 ts_tablet_manager.cc:1939] T d6664554cdae48f1b66ec1128c8d0f44 P cfd7a2d3199942488cace531d1903c76: Deleting consensus metadata
I20260618 22:27:46.740438 18766 catalog_manager.cc:4841] TS cfd7a2d3199942488cace531d1903c76 (127.10.242.194:37169): tablet d6664554cdae48f1b66ec1128c8d0f44 (table test-table [id=ff3c16bc01b84da3a6d7792f7d06eb90]) successfully deleted
I20260618 22:27:46.741598 19356 ts_tablet_manager.cc:1918] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:27:46.741735 19356 log.cc:1196] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/d6664554cdae48f1b66ec1128c8d0f44
I20260618 22:27:46.742148 19356 ts_tablet_manager.cc:1939] T d6664554cdae48f1b66ec1128c8d0f44 P bdde0480a48d43c99b6fd50ca6bd9ce7: Deleting consensus metadata
I20260618 22:27:46.743213 18764 catalog_manager.cc:4841] TS bdde0480a48d43c99b6fd50ca6bd9ce7 (127.10.242.193:43591): tablet d6664554cdae48f1b66ec1128c8d0f44 (table test-table [id=ff3c16bc01b84da3a6d7792f7d06eb90]) successfully deleted
Jun 18 22:27:46 dist-test-slave-0956 krb5kdc[18734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.193@KRBTEST.COM
I20260618 22:27:46.752885 19376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:46.729834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37470 (local address 127.10.242.193:43591)
0618 22:27:46.730036 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:46.730043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:46.737586 (+ 7543us) server_negotiation.cc:407] Connection header received
0618 22:27:46.737874 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:46.737879 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:46.737955 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:46.738024 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:46.739720 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:46.740825 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:46.742340 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:46.742798 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:46.742929 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:46.747000 (+ 4071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:46.747017 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:46.747033 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:46.747066 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:46.750023 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:46.750859 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:46.750867 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:46.750869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:46.751001 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:46.751372 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:46.751376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:46.751378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:46.751715 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:46.751937 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:46.752269 (+ 332us) server_negotiation.cc:299] Negotiation successful
0618 22:27:46.752459 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:27:46.753643 18981 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:46.901221 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 18850
I20260618 22:27:46.920118 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 18990
Jun 18 22:27:46 dist-test-slave-0956 krb5kdc[18734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.194@KRBTEST.COM
I20260618 22:27:46.936309 19394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:46.920883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47896 (local address 127.10.242.194:37169)
0618 22:27:46.921358 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:46.921363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:46.921379 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:27:46.921430 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:46.921434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:46.921491 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:46.921577 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:46.923036 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:46.923795 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:46.925567 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:46.925993 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:46.926082 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:46.930657 (+ 4575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:46.930671 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:46.930687 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:46.930727 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:46.933668 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:46.934337 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:46.934344 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:46.934346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:46.934477 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:46.934836 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:46.934840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:46.934842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:46.935173 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:46.935467 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:46.935776 (+ 309us) server_negotiation.cc:299] Negotiation successful
0618 22:27:46.935984 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":280,"threads_started":1}
I20260618 22:27:46.936885 19120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:47.036226 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 18990
I20260618 22:27:47.049679 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 19129
Jun 18 22:27:47 dist-test-slave-0956 krb5kdc[18734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.195@KRBTEST.COM
I20260618 22:27:47.063733 19386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:47.050343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41574 (local address 127.10.242.195:36539)
0618 22:27:47.050488 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:47.050492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:47.050557 (+ 65us) server_negotiation.cc:407] Connection header received
0618 22:27:47.050727 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:47.050731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:47.050803 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:47.050889 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:47.052228 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:47.052974 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:47.054328 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:47.054714 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:47.054809 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:47.058904 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:47.058920 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:47.058935 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:47.058969 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:47.061653 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:47.062215 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:47.062222 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:47.062225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:47.062350 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:47.062629 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:47.062632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:47.062634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:47.062952 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:47.063170 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:47.063345 (+ 175us) server_negotiation.cc:299] Negotiation successful
0618 22:27:47.063498 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:27:47.064343 19259 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:47.185305 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 19129
I20260618 22:27:47.201980 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 18747
I20260618 22:27:47.203081 18819 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:47.313549 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 18747
2026-06-18T22:27:47Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5627 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:27:47 dist-test-slave-0956 krb5kdc[19400](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:27:47 dist-test-slave-0956 krb5kdc[19400](info): set up 2 sockets
Jun 18 22:27:47 dist-test-slave-0956 krb5kdc[19400](info): commencing operation
krb5kdc: starting...
W20260618 22:27:48.317687 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.965s user 0.002s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:27:48 dist-test-slave-0956 krb5kdc[19400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:27:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:27:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:27:48.528860 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:33895
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:33895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:48.842213 19413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:48.842509 19413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:48.842638 19413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:48.850262 19413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:27:48.850405 19413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:48.850485 19413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:48.850566 19413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:27:48.850630 19413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:27:48.861783 19413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:33895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:33895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:48.863008 19413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:48.864220 19413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:48.875655 19418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:48.875765 19419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:48.875777 19421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:48.876302 19413 server_base.cc:1034] running on GCE node
I20260618 22:27:48.877127 19413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:48.878625 19413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:48.879851 19413 hybrid_clock.cc:648] HybridClock initialized: now 1781821668879795 us; error 44 us; skew 500 ppm
I20260618 22:27:48.880201 19413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-18T22:27:48Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1636 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:27:49 dist-test-slave-0956 krb5kdc[19427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:27:49 dist-test-slave-0956 krb5kdc[19427](info): set up 2 sockets
Jun 18 22:27:49 dist-test-slave-0956 krb5kdc[19427](info): commencing operation
krb5kdc: starting...
W20260618 22:27:49.958693 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.970s user 0.005s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:27:50 dist-test-slave-0956 krb5kdc[19427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:27:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:27:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:27:50.171891 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:45263
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:45263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:50.477865 19440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:50.478158 19440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:50.478273 19440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:50.485426 19440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:27:50.485530 19440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:50.485584 19440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:50.485661 19440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:27:50.485721 19440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:27:50.495673 19440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:45263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:45263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:50.497017 19440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:50.498303 19440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:50.509419 19446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:50.509482 19448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:50.509871 19445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:50.510251 19440 server_base.cc:1034] running on GCE node
I20260618 22:27:50.510936 19440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:50.512253 19440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:50.513469 19440 hybrid_clock.cc:648] HybridClock initialized: now 1781821670513422 us; error 30 us; skew 500 ppm
I20260618 22:27:50.513828 19440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:50 dist-test-slave-0956 krb5kdc[19427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821670, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:50.516959 19440 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:27:50.518435 19440 webserver.cc:460] Webserver started at http://127.10.242.254:46003/ using document root <none> and password file <none>
I20260618 22:27:50.519091 19440 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:50.519212 19440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:50.519498 19440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:50.522418 19440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "3a869c2db3bc45938172c5a0f9aaa86f"
format_stamp: "Formatted at 2026-06-18 22:27:50 on dist-test-slave-0956"
server_key: "5653ecb753253a53ce0b27cf9d38e38e"
server_key_iv: "70a8a51937a12f73a9a1b72be5eec3e7"
server_key_version: "encryptionkey@0"
I20260618 22:27:50.523145 19440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "3a869c2db3bc45938172c5a0f9aaa86f"
format_stamp: "Formatted at 2026-06-18 22:27:50 on dist-test-slave-0956"
server_key: "5653ecb753253a53ce0b27cf9d38e38e"
server_key_iv: "70a8a51937a12f73a9a1b72be5eec3e7"
server_key_version: "encryptionkey@0"
I20260618 22:27:50.528162 19440 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260618 22:27:50.531494 19455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:50.532809 19440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:27:50.532966 19440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "3a869c2db3bc45938172c5a0f9aaa86f"
format_stamp: "Formatted at 2026-06-18 22:27:50 on dist-test-slave-0956"
server_key: "5653ecb753253a53ce0b27cf9d38e38e"
server_key_iv: "70a8a51937a12f73a9a1b72be5eec3e7"
server_key_version: "encryptionkey@0"
I20260618 22:27:50.533186 19440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:50.553228 19440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:50.557337 19440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:50.557602 19440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:50.574991 19440 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:45263
I20260618 22:27:50.575012 19506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:45263 every 8 connection(s)
I20260618 22:27:50.576354 19440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:27:50.579982 19507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:27:50.580675 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 19440
I20260618 22:27:50.580896 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:27:50.581513 11211 external_mini_cluster.cc:1428] Setting key 7c79c69d790f1079e4210de5b712c9a4
I20260618 22:27:50.593287 19507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f: Bootstrap starting.
Jun 18 22:27:50 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:50.596974 19507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f: Neither blocks nor log segments found. Creating new log.
I20260618 22:27:50.598148 19507 log.cc:824] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:50.601292 19507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f: No bootstrap required, opened a new log
I20260618 22:27:50.604331 19510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:50.583366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33308 (local address 127.10.242.254:45263)
0618 22:27:50.584100 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:50.584111 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:50.584152 (+ 41us) server_negotiation.cc:407] Connection header received
0618 22:27:50.585810 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:50.585835 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:50.586236 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:50.586582 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:50.588262 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:50.589660 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:50.591405 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:50.592062 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:50.592194 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:50.596874 (+ 4680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:50.596885 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:50.596900 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:50.596946 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:50.600271 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:50.601105 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:50.601115 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:50.601123 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:50.601291 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:50.601758 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:50.601761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:50.601763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:50.602258 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:50.602455 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:50.602952 (+ 497us) server_negotiation.cc:299] Negotiation successful
0618 22:27:50.603294 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":316,"threads_started":1}
I20260618 22:27:50.608963 19507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a869c2db3bc45938172c5a0f9aaa86f" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45263 } }
I20260618 22:27:50.609342 19507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:27:50.609445 19507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a869c2db3bc45938172c5a0f9aaa86f, State: Initialized, Role: FOLLOWER
I20260618 22:27:50.609997 19507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a869c2db3bc45938172c5a0f9aaa86f" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45263 } }
I20260618 22:27:50.610216 19507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:27:50.610342 19507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:27:50.610493 19507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:27:50.612565 19507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a869c2db3bc45938172c5a0f9aaa86f" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45263 } }
I20260618 22:27:50.613036 19507 leader_election.cc:302] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a869c2db3bc45938172c5a0f9aaa86f; no voters:
I20260618 22:27:50.613646 19507 leader_election.cc:288] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:27:50.613838 19512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:27:50.614774 19512 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [term 1 LEADER]: Becoming Leader. State: Replica: 3a869c2db3bc45938172c5a0f9aaa86f, State: Running, Role: LEADER
I20260618 22:27:50.615227 19512 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a869c2db3bc45938172c5a0f9aaa86f" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45263 } }
I20260618 22:27:50.615741 19507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:27:50.617902 19514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a869c2db3bc45938172c5a0f9aaa86f. Latest consensus state: current_term: 1 leader_uuid: "3a869c2db3bc45938172c5a0f9aaa86f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a869c2db3bc45938172c5a0f9aaa86f" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45263 } } }
I20260618 22:27:50.617879 19513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a869c2db3bc45938172c5a0f9aaa86f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a869c2db3bc45938172c5a0f9aaa86f" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45263 } } }
I20260618 22:27:50.618285 19514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:50.618290 19513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:50.619880 19520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:27:50.624328 19520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:27:50.632871 19520 catalog_manager.cc:1347] Generated new cluster ID: e44914fc73eb44adb188e8d0dc0d83ca
I20260618 22:27:50.632987 19520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:27:50.643921 19520 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:27:50.645275 19520 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:27:50.651993 19520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f: Generated new TSK 0
I20260618 22:27:50.652864 19520 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
I20260618 22:27:50.710005 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45263
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:51.036176 19533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:51.036466 19533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:51.036585 19533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:51.043982 19533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:51.044104 19533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:51.044257 19533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:27:51.054004 19533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45263
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:51.055347 19533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:51.056708 19533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:51.070494 19541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:51.070469 19538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:51.070578 19539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:51.071138 19533 server_base.cc:1034] running on GCE node
I20260618 22:27:51.071764 19533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:51.072721 19533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:51.073951 19533 hybrid_clock.cc:648] HybridClock initialized: now 1781821671073889 us; error 57 us; skew 500 ppm
I20260618 22:27:51.074268 19533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:51 dist-test-slave-0956 krb5kdc[19427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821671, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:51.077513 19533 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:27:51.079035 19533 webserver.cc:460] Webserver started at http://127.10.242.193:35547/ using document root <none> and password file <none>
I20260618 22:27:51.079731 19533 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:51.079852 19533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:51.080153 19533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:51.083173 19533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "95dcd8fb53e44c9da55fe89ba7c7cd70"
format_stamp: "Formatted at 2026-06-18 22:27:51 on dist-test-slave-0956"
server_key: "a22b22b8e196f985828428eb12b250cb"
server_key_iv: "bc7ba244f5c745cfa56c22d6c50be3d7"
server_key_version: "encryptionkey@0"
I20260618 22:27:51.083995 19533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "95dcd8fb53e44c9da55fe89ba7c7cd70"
format_stamp: "Formatted at 2026-06-18 22:27:51 on dist-test-slave-0956"
server_key: "a22b22b8e196f985828428eb12b250cb"
server_key_iv: "bc7ba244f5c745cfa56c22d6c50be3d7"
server_key_version: "encryptionkey@0"
I20260618 22:27:51.089308 19533 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260618 22:27:51.092841 19548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:51.094244 19533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:27:51.094400 19533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "95dcd8fb53e44c9da55fe89ba7c7cd70"
format_stamp: "Formatted at 2026-06-18 22:27:51 on dist-test-slave-0956"
server_key: "a22b22b8e196f985828428eb12b250cb"
server_key_iv: "bc7ba244f5c745cfa56c22d6c50be3d7"
server_key_version: "encryptionkey@0"
I20260618 22:27:51.094594 19533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:51.122354 19533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:51.126619 19533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:51.126893 19533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:51.127835 19533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:51.129374 19533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:51.129478 19533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:51.129611 19533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:51.129662 19533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:51.159983 19533 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:36723
I20260618 22:27:51.160071 19660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:36723 every 8 connection(s)
I20260618 22:27:51.161374 19533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:27:51.170679 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 19533
I20260618 22:27:51.170902 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:27:51.171650 11211 external_mini_cluster.cc:1428] Setting key 88010892cbbcd3afa8ae02c138987ae1
Jun 18 22:27:51 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821671, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:51.181600 19663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:51.164053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:57573 (local address 127.10.242.254:45263)
0618 22:27:51.164513 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:51.164520 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:51.165661 (+ 1141us) server_negotiation.cc:407] Connection header received
0618 22:27:51.167599 (+ 1938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:51.167604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:51.167687 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:51.167785 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:51.169483 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:51.170176 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:51.172472 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:51.172973 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:51.173130 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:51.176366 (+ 3236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:51.176371 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:51.176374 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:51.176407 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:51.178739 (+ 2332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:51.179425 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:51.179429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:51.179431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:51.179538 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:51.179935 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:51.179938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:51.179940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:51.180185 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:51.180315 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:51.181004 (+ 689us) server_negotiation.cc:299] Negotiation successful
0618 22:27:51.181244 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":208,"threads_started":1}
I20260618 22:27:51.182575 19664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:51.164638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:57573)
0618 22:27:51.165483 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:51.165530 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:51.167307 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:51.168054 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:51.168063 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:51.168601 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:51.169302 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:51.169322 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:51.170345 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:51.170351 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:51.172309 (+ 1958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:51.172321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:51.173226 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:51.173232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:51.173437 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:51.174060 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:51.174083 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:51.176141 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:51.178872 (+ 2731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:51.178882 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:51.178900 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:51.179304 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:51.179632 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:51.179637 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:51.179641 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:51.179827 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:51.180302 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:51.180310 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:51.180642 (+ 332us) client_negotiation.cc:769] Sending connection context
0618 22:27:51.180882 (+ 240us) client_negotiation.cc:240] Negotiation successful
0618 22:27:51.181431 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":343,"threads_started":1}
I20260618 22:27:51.184137 19661 heartbeater.cc:347] Connected to a master server at 127.10.242.254:45263
I20260618 22:27:51.184430 19661 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:51.185166 19661 heartbeater.cc:511] Master 127.10.242.254:45263 requested a full tablet report, sending...
I20260618 22:27:51.187463 19472 ts_manager.cc:194] Registered new tserver with Master: 95dcd8fb53e44c9da55fe89ba7c7cd70 (127.10.242.193:36723)
I20260618 22:27:51.189581 19472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:57573
WARNING: no policy specified for kudu/127.10.242.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
I20260618 22:27:51.216919 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45263
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:51.529951 19667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:51.530217 19667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:51.530339 19667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:51.537532 19667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:51.537654 19667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:51.537806 19667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:27:51.547453 19667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45263
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:51.548729 19667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:51.550022 19667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:51.562673 19673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:51.562649 19675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:51.562721 19672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:51.563361 19667 server_base.cc:1034] running on GCE node
I20260618 22:27:51.563954 19667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:51.564900 19667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:51.566124 19667 hybrid_clock.cc:648] HybridClock initialized: now 1781821671566060 us; error 57 us; skew 500 ppm
I20260618 22:27:51.566483 19667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:51 dist-test-slave-0956 krb5kdc[19427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821671, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:51.569604 19667 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:27:51.571116 19667 webserver.cc:460] Webserver started at http://127.10.242.194:43487/ using document root <none> and password file <none>
I20260618 22:27:51.571821 19667 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:51.571941 19667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:51.572252 19667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:51.575593 19667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "4f52193b4fe34c62880714a2b64625c1"
format_stamp: "Formatted at 2026-06-18 22:27:51 on dist-test-slave-0956"
server_key: "b040c0da6d16db9332d540c44d09f343"
server_key_iv: "885c45cdd987af4dd7fb529351da1985"
server_key_version: "encryptionkey@0"
I20260618 22:27:51.576572 19667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "4f52193b4fe34c62880714a2b64625c1"
format_stamp: "Formatted at 2026-06-18 22:27:51 on dist-test-slave-0956"
server_key: "b040c0da6d16db9332d540c44d09f343"
server_key_iv: "885c45cdd987af4dd7fb529351da1985"
server_key_version: "encryptionkey@0"
I20260618 22:27:51.582978 19667 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.001s
I20260618 22:27:51.587447 19682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:51.589203 19667 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260618 22:27:51.589398 19667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "4f52193b4fe34c62880714a2b64625c1"
format_stamp: "Formatted at 2026-06-18 22:27:51 on dist-test-slave-0956"
server_key: "b040c0da6d16db9332d540c44d09f343"
server_key_iv: "885c45cdd987af4dd7fb529351da1985"
server_key_version: "encryptionkey@0"
I20260618 22:27:51.589627 19667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:51.616801 19667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:51.621253 19667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:51.621498 19667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:51.622417 19667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:51.623725 19667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:51.623795 19667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:51.623876 19667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:51.623919 19667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:51.654489 19667 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:36271
I20260618 22:27:51.654563 19794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:36271 every 8 connection(s)
I20260618 22:27:51.655746 19667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:27:51.657184 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 19667
I20260618 22:27:51.657394 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:27:51.657804 11211 external_mini_cluster.cc:1428] Setting key 9a6aeaf0473cf1b918ff6aee6723d969
Jun 18 22:27:51 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821671, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:51.677416 19663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:51.658386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:53681 (local address 127.10.242.254:45263)
0618 22:27:51.658573 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:51.658579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:51.659904 (+ 1325us) server_negotiation.cc:407] Connection header received
0618 22:27:51.662408 (+ 2504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:51.662412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:51.662478 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:51.662566 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:51.664480 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:51.665459 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:51.667555 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:51.668223 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:51.668348 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:51.672077 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:51.672082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:51.672085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:51.672118 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:51.674515 (+ 2397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:51.675213 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:51.675219 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:51.675221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:51.675342 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:51.675749 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:51.675753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:51.675757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:51.676024 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:51.676147 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:51.676821 (+ 674us) server_negotiation.cc:299] Negotiation successful
0618 22:27:51.677043 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:27:51.678435 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:51.658961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:53681)
0618 22:27:51.659749 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:51.659800 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:51.662089 (+ 2289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:51.662822 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:51.662834 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:51.663520 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:51.664297 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:51.664322 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:51.665607 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:51.665612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:51.667381 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:51.667401 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:51.668446 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:51.668452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:51.668652 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:51.669365 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:51.669393 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:51.671854 (+ 2461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:51.674653 (+ 2799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:51.674663 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:51.674683 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:51.675099 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:51.675444 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:51.675449 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:51.675453 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:51.675639 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:51.676141 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:51.676149 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:51.676481 (+ 332us) client_negotiation.cc:769] Sending connection context
0618 22:27:51.676726 (+ 245us) client_negotiation.cc:240] Negotiation successful
0618 22:27:51.677305 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":317,"threads_started":1}
WARNING: no policy specified for kudu/127.10.242.195@KRBTEST.COM; defaulting to no policy
I20260618 22:27:51.679895 19795 heartbeater.cc:347] Connected to a master server at 127.10.242.254:45263
I20260618 22:27:51.680179 19795 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:51.680889 19795 heartbeater.cc:511] Master 127.10.242.254:45263 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.195@KRBTEST.COM" created.
I20260618 22:27:51.682313 19472 ts_manager.cc:194] Registered new tserver with Master: 4f52193b4fe34c62880714a2b64625c1 (127.10.242.194:36271)
I20260618 22:27:51.683075 19472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:53681
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
I20260618 22:27:51.705803 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45263
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:52.027321 19800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:52.027593 19800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:52.027686 19800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:52.034795 19800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:52.034916 19800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:52.035046 19800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:27:52.044649 19800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45263
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:52.045876 19800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:52.047098 19800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:52.059162 19806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:52.059134 19808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:52.059406 19805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:52.059787 19800 server_base.cc:1034] running on GCE node
I20260618 22:27:52.060341 19800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:52.061256 19800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:52.062464 19800 hybrid_clock.cc:648] HybridClock initialized: now 1781821672062399 us; error 60 us; skew 500 ppm
I20260618 22:27:52.062765 19800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:52 dist-test-slave-0956 krb5kdc[19427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821672, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:52.065933 19800 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:27:52.067413 19800 webserver.cc:460] Webserver started at http://127.10.242.195:37497/ using document root <none> and password file <none>
I20260618 22:27:52.068084 19800 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:52.068197 19800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:52.068476 19800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:27:52.071326 19800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "d365c27ded4544339e8a132a48750e89"
format_stamp: "Formatted at 2026-06-18 22:27:52 on dist-test-slave-0956"
server_key: "5a09b7c5d49ca9d0ef745ded473c24e8"
server_key_iv: "7387d9296d94e957afb1805af84376ea"
server_key_version: "encryptionkey@0"
I20260618 22:27:52.072045 19800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "d365c27ded4544339e8a132a48750e89"
format_stamp: "Formatted at 2026-06-18 22:27:52 on dist-test-slave-0956"
server_key: "5a09b7c5d49ca9d0ef745ded473c24e8"
server_key_iv: "7387d9296d94e957afb1805af84376ea"
server_key_version: "encryptionkey@0"
I20260618 22:27:52.076964 19800 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260618 22:27:52.080279 19815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:52.081825 19800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:27:52.082010 19800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "d365c27ded4544339e8a132a48750e89"
format_stamp: "Formatted at 2026-06-18 22:27:52 on dist-test-slave-0956"
server_key: "5a09b7c5d49ca9d0ef745ded473c24e8"
server_key_iv: "7387d9296d94e957afb1805af84376ea"
server_key_version: "encryptionkey@0"
I20260618 22:27:52.082206 19800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:52.099140 19800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:52.103075 19800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:52.103345 19800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:52.104270 19800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:52.105616 19800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:52.105695 19800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:52.105814 19800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:52.105866 19800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:52.136564 19800 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:39419
I20260618 22:27:52.136610 19927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:39419 every 8 connection(s)
I20260618 22:27:52.137825 19800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:27:52.145601 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 19800
I20260618 22:27:52.145762 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:27:52.146160 11211 external_mini_cluster.cc:1428] Setting key 70239deffeb683fac55e77c76d160ec2
Jun 18 22:27:52 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821672, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:27:52.156030 19663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:52.140175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:46613 (local address 127.10.242.254:45263)
0618 22:27:52.140327 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:52.140331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:52.141487 (+ 1156us) server_negotiation.cc:407] Connection header received
0618 22:27:52.143240 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:52.143246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:52.143346 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:52.143430 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:52.144884 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:52.145645 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:52.147314 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:52.147859 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:52.147984 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:52.151330 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:52.151334 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:52.151338 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:52.151368 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:52.153644 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:52.154242 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:52.154245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:52.154248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:52.154360 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:52.154677 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:52.154687 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:52.154689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:52.154960 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:52.155063 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:52.155610 (+ 547us) server_negotiation.cc:299] Negotiation successful
0618 22:27:52.155760 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:27:52.156906 19930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:52.140639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.195:46613)
0618 22:27:52.141336 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:52.141372 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:52.142982 (+ 1610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:52.143672 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:52.143683 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:52.144169 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:52.144722 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:52.144738 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:52.145844 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:52.145850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:52.147188 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:52.147200 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:52.148094 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:52.148101 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:52.148380 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:52.149142 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:52.149168 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:52.151163 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:27:52.153751 (+ 2588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:52.153757 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:52.153769 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:52.154157 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:52.154436 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:27:52.154439 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:27:52.154442 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:27:52.154605 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:27:52.155047 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:27:52.155052 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:27:52.155338 (+ 286us) client_negotiation.cc:769] Sending connection context
0618 22:27:52.155548 (+ 210us) client_negotiation.cc:240] Negotiation successful
0618 22:27:52.155902 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":250,"threads_started":1}
I20260618 22:27:52.158286 19928 heartbeater.cc:347] Connected to a master server at 127.10.242.254:45263
I20260618 22:27:52.158555 19928 heartbeater.cc:464] Registering TS with master...
I20260618 22:27:52.159209 19928 heartbeater.cc:511] Master 127.10.242.254:45263 requested a full tablet report, sending...
I20260618 22:27:52.160791 19472 ts_manager.cc:194] Registered new tserver with Master: d365c27ded4544339e8a132a48750e89 (127.10.242.195:39419)
I20260618 22:27:52.161479 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:27:52.161795 19472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:46613
WARNING: no policy specified for oryx/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.193@KRBTEST.COM" created.
I20260618 22:27:52.193346 19661 heartbeater.cc:503] Master 127.10.242.254:45263 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.193.keytab.
Entry for principal oryx/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.193.keytab.
WARNING: no policy specified for oryx/127.10.242.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.194.keytab.
Entry for principal oryx/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.194.keytab.
WARNING: no policy specified for oryx/127.10.242.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.195.keytab.
Entry for principal oryx/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.195.keytab.
I20260618 22:27:52.283493 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 19533
Jun 18 22:27:52 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:27:52.299196 19938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:52.284591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60266 (local address 127.10.242.193:36723)
0618 22:27:52.285208 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:52.285215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:52.285242 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:27:52.285299 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:52.285305 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:52.285533 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:52.285677 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:52.287069 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:52.288040 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:52.289328 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:52.289886 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:52.289983 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:52.294005 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:52.294019 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:52.294035 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:52.294070 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:52.296761 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:52.297456 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:52.297463 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:52.297466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:52.297589 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:52.297894 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:52.297897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:52.297899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:52.298233 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:52.298490 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:52.298700 (+ 210us) server_negotiation.cc:299] Negotiation successful
0618 22:27:52.298919 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":287,"threads_started":1}
I20260618 22:27:52.300423 19656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:52.382421 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 19533
I20260618 22:27:52.395887 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 19667
Jun 18 22:27:52 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.194@KRBTEST.COM
I20260618 22:27:52.410632 19940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:52.396654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59588 (local address 127.10.242.194:36271)
0618 22:27:52.397225 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:52.397233 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:52.397258 (+ 25us) server_negotiation.cc:407] Connection header received
0618 22:27:52.397319 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:52.397326 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:52.397540 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:52.397668 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:52.398822 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:52.399819 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:52.401102 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:52.401661 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:52.401763 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:52.405498 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:52.405509 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:52.405523 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:52.405555 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:52.408210 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:52.408771 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:52.408778 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:52.408781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:52.408893 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:52.409354 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:52.409358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:52.409360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:52.409698 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:52.409941 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:52.410136 (+ 195us) server_negotiation.cc:299] Negotiation successful
0618 22:27:52.410327 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":281,"threads_started":1}
I20260618 22:27:52.411911 19790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:52.492496 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 19667
I20260618 22:27:52.504401 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 19800
Jun 18 22:27:52 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.195@KRBTEST.COM
I20260618 22:27:52.519560 19942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:52.505258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44522 (local address 127.10.242.195:39419)
0618 22:27:52.505826 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:52.505833 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:52.505863 (+ 30us) server_negotiation.cc:407] Connection header received
0618 22:27:52.505921 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:52.505936 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:52.506141 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:52.506269 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:52.507434 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:52.508452 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:52.509733 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:52.510285 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:52.510387 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:52.514397 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:52.514412 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:52.514429 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:52.514466 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:52.517159 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:52.517784 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:52.517791 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:52.517794 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:52.517905 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:52.518198 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:52.518201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:52.518203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:52.518523 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:52.518803 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:52.519059 (+ 256us) server_negotiation.cc:299] Negotiation successful
0618 22:27:52.519272 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":266,"threads_started":1}
I20260618 22:27:52.520751 19923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:52.601153 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 19800
I20260618 22:27:52.612953 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 19440
I20260618 22:27:52.614096 19502 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:27:52.714164 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 19440
I20260618 22:27:52.730476 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:45263
--webserver_interface=127.10.242.254
--webserver_port=46003
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:45263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:27:53.053992 19945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:53.054287 19945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:53.054420 19945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:53.061815 19945 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:27:53.061930 19945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:53.062000 19945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:53.062078 19945 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:27:53.062137 19945 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:27:53.072072 19945 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:45263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:45263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=46003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:53.073414 19945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:53.074697 19945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:53.085968 19950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:53.086027 19951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:53.086287 19953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:53.086745 19945 server_base.cc:1034] running on GCE node
I20260618 22:27:53.087571 19945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:53.089037 19945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:53.090269 19945 hybrid_clock.cc:648] HybridClock initialized: now 1781821673090208 us; error 47 us; skew 500 ppm
I20260618 22:27:53.090641 19945 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:53 dist-test-slave-0956 krb5kdc[19427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821673, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:53.093775 19945 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:27:53.095321 19945 webserver.cc:460] Webserver started at http://127.10.242.254:46003/ using document root <none> and password file <none>
I20260618 22:27:53.096024 19945 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:53.096139 19945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:53.101779 19945 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260618 22:27:53.104696 19960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:53.106084 19945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:27:53.106282 19945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "3a869c2db3bc45938172c5a0f9aaa86f"
format_stamp: "Formatted at 2026-06-18 22:27:50 on dist-test-slave-0956"
server_key: "5653ecb753253a53ce0b27cf9d38e38e"
server_key_iv: "70a8a51937a12f73a9a1b72be5eec3e7"
server_key_version: "encryptionkey@0"
I20260618 22:27:53.107229 19945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:53.124583 19945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:53.128429 19945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:53.128666 19945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:53.144556 19945 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:45263
I20260618 22:27:53.144564 20011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:45263 every 8 connection(s)
I20260618 22:27:53.146010 19945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:27:53.150324 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 19945
I20260618 22:27:53.151055 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:36723
--local_ip_for_outbound_sockets=127.10.242.193
--tserver_master_addrs=127.10.242.254:45263
--webserver_port=35547
--webserver_interface=127.10.242.193
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.242.193
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:27:53.152717 20012 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:27:53.155895 20012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f: Bootstrap starting.
I20260618 22:27:53.182090 20012 log.cc:824] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f: Log is configured to *not* fsync() on all Append() calls
I20260618 22:27:53.193270 20012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:27:53.193894 20012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f: Bootstrap complete.
I20260618 22:27:53.206921 20012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a869c2db3bc45938172c5a0f9aaa86f" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45263 } }
I20260618 22:27:53.208163 20012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a869c2db3bc45938172c5a0f9aaa86f, State: Initialized, Role: FOLLOWER
I20260618 22:27:53.208953 20012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a869c2db3bc45938172c5a0f9aaa86f" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45263 } }
I20260618 22:27:53.209259 20012 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:27:53.209378 20012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:27:53.209576 20012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:27:53.215584 20012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a869c2db3bc45938172c5a0f9aaa86f" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45263 } }
I20260618 22:27:53.216336 20012 leader_election.cc:302] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a869c2db3bc45938172c5a0f9aaa86f; no voters:
I20260618 22:27:53.217070 20012 leader_election.cc:288] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260618 22:27:53.217243 20016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [term 2 FOLLOWER]: Leader election won for term 2
I20260618 22:27:53.219300 20012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:27:53.224264 20016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [term 2 LEADER]: Becoming Leader. State: Replica: 3a869c2db3bc45938172c5a0f9aaa86f, State: Running, Role: LEADER
I20260618 22:27:53.224908 20016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a869c2db3bc45938172c5a0f9aaa86f" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45263 } }
I20260618 22:27:53.227542 20017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3a869c2db3bc45938172c5a0f9aaa86f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a869c2db3bc45938172c5a0f9aaa86f" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45263 } } }
I20260618 22:27:53.227546 20018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a869c2db3bc45938172c5a0f9aaa86f. Latest consensus state: current_term: 2 leader_uuid: "3a869c2db3bc45938172c5a0f9aaa86f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a869c2db3bc45938172c5a0f9aaa86f" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45263 } } }
I20260618 22:27:53.227968 20018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:53.227967 20017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f [sys.catalog]: This master's current role is: LEADER
I20260618 22:27:53.238372 20026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:27:53.243746 20026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:27:53.245329 20026 catalog_manager.cc:1259] Loaded cluster ID: e44914fc73eb44adb188e8d0dc0d83ca
I20260618 22:27:53.245523 20026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:27:53.249868 20026 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:27:53.251535 20026 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3a869c2db3bc45938172c5a0f9aaa86f: Loaded TSK: 0
I20260618 22:27:53.253585 20026 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260618 22:27:53.534246 20014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:53.534526 20014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:53.534636 20014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:53.542119 20014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:53.542248 20014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:53.542388 20014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:27:53.552619 20014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.193.keytab
--principal=oryx/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:36723
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=35547
--tserver_master_addrs=127.10.242.254:45263
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:53.554046 20014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:53.555455 20014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:53.570468 20039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:53.571110 20041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:53.571117 20038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:53.571611 20014 server_base.cc:1034] running on GCE node
I20260618 22:27:53.572499 20014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:53.573909 20014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:53.575166 20014 hybrid_clock.cc:648] HybridClock initialized: now 1781821673575095 us; error 66 us; skew 500 ppm
I20260618 22:27:53.575562 20014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:53 dist-test-slave-0956 krb5kdc[19427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821673, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:53.579675 20014 init.cc:377] Logged in from keytab as oryx/127.10.242.193@KRBTEST.COM (short username oryx)
I20260618 22:27:53.581404 20014 webserver.cc:460] Webserver started at http://127.10.242.193:35547/ using document root <none> and password file <none>
I20260618 22:27:53.582141 20014 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:53.582271 20014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:53.588265 20014 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260618 22:27:53.591526 20048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:53.593192 20014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:27:53.593371 20014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "95dcd8fb53e44c9da55fe89ba7c7cd70"
format_stamp: "Formatted at 2026-06-18 22:27:51 on dist-test-slave-0956"
server_key: "a22b22b8e196f985828428eb12b250cb"
server_key_iv: "bc7ba244f5c745cfa56c22d6c50be3d7"
server_key_version: "encryptionkey@0"
I20260618 22:27:53.594385 20014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:53.631518 20014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:53.636320 20014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:53.636598 20014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:53.637639 20014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:53.639070 20014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:53.639165 20014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:53.639278 20014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:53.639344 20014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:53.670439 20014 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:36723
I20260618 22:27:53.670512 20160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:36723 every 8 connection(s)
I20260618 22:27:53.671978 20014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:27:53.676872 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 20014
I20260618 22:27:53.677461 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:36271
--local_ip_for_outbound_sockets=127.10.242.194
--tserver_master_addrs=127.10.242.254:45263
--webserver_port=43487
--webserver_interface=127.10.242.194
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.242.194
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:27:53 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:53 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:53.691736 20164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:53.675120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:46089)
0618 22:27:53.676005 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:53.676060 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:53.677899 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:53.679676 (+ 1777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:53.679684 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:53.680224 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:53.680802 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:53.680822 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:53.683078 (+ 2256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:53.683083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:53.684513 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:53.684528 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:53.685727 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:53.685735 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:53.686037 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:53.686707 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:53.686731 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:53.690323 (+ 3592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":342,"threads_started":1}
W20260618 22:27:53.691805 20163 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:53.674990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:46089 (local address 127.10.242.254:45263)
0618 22:27:53.675657 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:53.675668 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:53.676208 (+ 540us) server_negotiation.cc:407] Connection header received
0618 22:27:53.678528 (+ 2320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:53.678547 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:53.679068 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:53.679502 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:53.681020 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:53.682870 (+ 1850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:53.684681 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:53.685451 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:53.685673 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:53.690733 (+ 5060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.193:46089: BlockingRecv error: recv got EOF from 127.10.242.193:46089 (error 108)
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":240,"threads_started":1}
W20260618 22:27:53.692782 20161 heartbeater.cc:644] Failed to heartbeat to 127.10.242.254:45263 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.242.254:45263: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Jun 18 22:27:53 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:53 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:53.701488 20164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:53.693422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:40719)
0618 22:27:53.693587 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:53.693607 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:53.693705 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:53.694044 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:53.694049 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:53.694404 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:53.694547 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:53.694556 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:53.695628 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:53.695634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:53.696592 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:53.696604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:53.697300 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:53.697305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:53.697478 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:53.698113 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:53.698133 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:53.701204 (+ 3071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260618 22:27:53.701627 20163 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:53.693619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:40719 (local address 127.10.242.254:45263)
0618 22:27:53.693790 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:53.693797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:53.693823 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:27:53.693867 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:53.693871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:53.693925 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:53.694001 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:53.694684 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:53.695483 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:53.696729 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:53.697164 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:53.697276 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:53.701416 (+ 4140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.193:40719: BlockingRecv error: recv got EOF from 127.10.242.193:40719 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 18 22:27:53 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:53 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:53.709836 20164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:53.702450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:46259)
0618 22:27:53.702600 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:53.702617 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:53.702708 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:53.703048 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:53.703053 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:53.703337 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:53.703487 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:53.703495 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:53.704510 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:53.704514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:53.705481 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:53.705493 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:53.706140 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:53.706145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:53.706299 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:53.706854 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:53.706870 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:53.709542 (+ 2672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260618 22:27:53.709981 20163 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:53.702624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:46259 (local address 127.10.242.254:45263)
0618 22:27:53.702789 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:53.702795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:53.702810 (+ 15us) server_negotiation.cc:407] Connection header received
0618 22:27:53.702852 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:53.702856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:53.702922 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:53.703007 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:53.703615 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:53.704365 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:53.705606 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:53.706013 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:53.706118 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:53.709752 (+ 3634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.193:46259: BlockingRecv error: recv got EOF from 127.10.242.193:46259 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260618 22:27:53.710388 20161 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260618 22:27:53.992882 20165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:53.993201 20165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:53.993302 20165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:54.000514 20165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:54.000627 20165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:54.000746 20165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:27:54.010329 20165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.194.keytab
--principal=oryx/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:36271
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=43487
--tserver_master_addrs=127.10.242.254:45263
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:54.011675 20165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:54.012964 20165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:54.025527 20173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:54.025586 20171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:54.025802 20170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:54.026342 20165 server_base.cc:1034] running on GCE node
I20260618 22:27:54.026962 20165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:54.027974 20165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:54.029161 20165 hybrid_clock.cc:648] HybridClock initialized: now 1781821674029105 us; error 56 us; skew 500 ppm
I20260618 22:27:54.029457 20165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:54 dist-test-slave-0956 krb5kdc[19427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821674, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:54.032449 20165 init.cc:377] Logged in from keytab as oryx/127.10.242.194@KRBTEST.COM (short username oryx)
I20260618 22:27:54.033980 20165 webserver.cc:460] Webserver started at http://127.10.242.194:43487/ using document root <none> and password file <none>
I20260618 22:27:54.034646 20165 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:54.034770 20165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:54.040207 20165 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260618 22:27:54.042841 20180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:54.044174 20165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:27:54.044324 20165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "4f52193b4fe34c62880714a2b64625c1"
format_stamp: "Formatted at 2026-06-18 22:27:51 on dist-test-slave-0956"
server_key: "b040c0da6d16db9332d540c44d09f343"
server_key_iv: "885c45cdd987af4dd7fb529351da1985"
server_key_version: "encryptionkey@0"
I20260618 22:27:54.045372 20165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:54.073922 20165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:54.077924 20165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:54.078189 20165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:54.079190 20165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:54.080534 20165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:54.080618 20165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:54.080691 20165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:54.080744 20165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:54.110399 20165 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:36271
I20260618 22:27:54.110499 20292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:36271 every 8 connection(s)
I20260618 22:27:54.111816 20165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:27:54.119277 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 20165
I20260618 22:27:54.119976 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:39419
--local_ip_for_outbound_sockets=127.10.242.195
--tserver_master_addrs=127.10.242.254:45263
--webserver_port=37497
--webserver_interface=127.10.242.195
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.242.195
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:27:54 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:54 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
W20260618 22:27:54.127919 20163 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:54.114411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:42467 (local address 127.10.242.254:45263)
0618 22:27:54.114568 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:54.114573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:54.115798 (+ 1225us) server_negotiation.cc:407] Connection header received
0618 22:27:54.117943 (+ 2145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:54.117949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:54.118031 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:54.118126 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:54.120030 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.120764 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.122512 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.122924 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.123042 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:54.127684 (+ 4642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.194:42467: BlockingRecv error: recv got EOF from 127.10.242.194:42467 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:27:54.128654 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:54.114967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:42467)
0618 22:27:54.115650 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:54.115691 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:54.117626 (+ 1935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:54.118409 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:54.118420 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:54.119151 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:54.119871 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:54.119890 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.120922 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.120927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:54.122360 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:54.122373 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.123151 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.123158 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:54.123391 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:54.124035 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:54.124057 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:54.127487 (+ 3430us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":265,"threads_started":1}
W20260618 22:27:54.129292 20293 heartbeater.cc:644] Failed to heartbeat to 127.10.242.254:45263 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.242.254:45263: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Jun 18 22:27:54 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:54 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:54.137434 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:54.129866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:46653)
0618 22:27:54.130044 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:54.130063 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:54.130147 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:54.130419 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:54.130422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:54.130698 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:54.130823 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:54.130831 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.131999 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.132003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:54.132975 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:54.132988 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.133809 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.133813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:54.133932 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:54.134502 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:54.134517 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:54.137238 (+ 2721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260618 22:27:54.137759 20163 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:54.129983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:46653 (local address 127.10.242.254:45263)
0618 22:27:54.130136 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:54.130142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:54.130161 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:27:54.130247 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:54.130252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:54.130319 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:54.130401 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:54.130941 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.131886 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.133118 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.133676 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.133799 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:54.137502 (+ 3703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.194:46653: BlockingRecv error: recv got EOF from 127.10.242.194:46653 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 18 22:27:54 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:54 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:54.146149 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:54.138179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:36055)
0618 22:27:54.138310 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:54.138329 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:54.138405 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:54.138696 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:54.138699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:54.138969 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:54.139103 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:54.139110 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.140414 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.140417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:54.141361 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:54.141372 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.142219 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.142223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:54.142341 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:54.142851 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:54.142866 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:54.145885 (+ 3019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260618 22:27:54.146327 20163 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:54.138280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:36055 (local address 127.10.242.254:45263)
0618 22:27:54.138435 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:54.138440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:54.138459 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:27:54.138513 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:54.138518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:54.138588 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:54.138691 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:54.139226 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.140286 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.141479 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.142106 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.142221 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:54.146102 (+ 3881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.194:36055: BlockingRecv error: recv got EOF from 127.10.242.194:36055 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260618 22:27:54.146627 20293 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260618 22:27:54.437891 20296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:27:54.438189 20296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:27:54.438330 20296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:27:54.445463 20296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:27:54.445585 20296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:27:54.445731 20296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:27:54.455451 20296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/krb5kdc/oryx_127.10.242.195.keytab
--principal=oryx/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:39419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=37497
--tserver_master_addrs=127.10.242.254:45263
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:27:54.456727 20296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:27:54.458041 20296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:27:54.470925 20302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:54.470852 20304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:27:54.470885 20301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:27:54.478649 20296 server_base.cc:1034] running on GCE node
I20260618 22:27:54.479337 20296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:27:54.480325 20296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:27:54.481520 20296 hybrid_clock.cc:648] HybridClock initialized: now 1781821674481461 us; error 52 us; skew 500 ppm
I20260618 22:27:54.481827 20296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:27:54 dist-test-slave-0956 krb5kdc[19427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821674, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:27:54.484863 20296 init.cc:377] Logged in from keytab as oryx/127.10.242.195@KRBTEST.COM (short username oryx)
I20260618 22:27:54.486454 20296 webserver.cc:460] Webserver started at http://127.10.242.195:37497/ using document root <none> and password file <none>
I20260618 22:27:54.487138 20296 fs_manager.cc:359] Metadata directory not provided
I20260618 22:27:54.487247 20296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:27:54.492506 20296 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:27:54.495158 20311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:54.496505 20296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:27:54.496649 20296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "d365c27ded4544339e8a132a48750e89"
format_stamp: "Formatted at 2026-06-18 22:27:52 on dist-test-slave-0956"
server_key: "5a09b7c5d49ca9d0ef745ded473c24e8"
server_key_iv: "7387d9296d94e957afb1805af84376ea"
server_key_version: "encryptionkey@0"
I20260618 22:27:54.497664 20296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:27:54.520495 20296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:27:54.524887 20296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:27:54.525180 20296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:27:54.526170 20296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:27:54.527477 20296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:27:54.527562 20296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:54.527659 20296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:27:54.527721 20296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:27:54.556686 20296 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:39419
I20260618 22:27:54.556751 20423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:39419 every 8 connection(s)
I20260618 22:27:54.558164 20296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:27:54.561067 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 20296
Jun 18 22:27:54 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:54 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
W20260618 22:27:54.575572 20163 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:54.560830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:59281 (local address 127.10.242.254:45263)
0618 22:27:54.561066 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:54.561072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:54.562370 (+ 1298us) server_negotiation.cc:407] Connection header received
0618 22:27:54.564874 (+ 2504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:54.564893 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:54.564981 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:54.565049 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:54.566929 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.567632 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.569650 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.570080 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.570171 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:54.575341 (+ 5170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.195:59281: BlockingRecv error: recv got EOF from 127.10.242.195:59281 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
I20260618 22:27:54.576238 20426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:54.561444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.195:59281)
0618 22:27:54.562214 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:54.562266 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:54.564559 (+ 2293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:54.565593 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:54.565605 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:54.566114 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:54.566773 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:54.566795 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.567774 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.567778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:54.569511 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:54.569526 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.570312 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.570321 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:54.570637 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:54.571371 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:54.571405 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:54.575126 (+ 3721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":298,"threads_started":1}
W20260618 22:27:54.577049 20424 heartbeater.cc:644] Failed to heartbeat to 127.10.242.254:45263 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.242.254:45263: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
I20260618 22:27:54.577685 20427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:54.561959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33310 (local address 127.10.242.254:45263)
0618 22:27:54.562914 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:54.562919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:54.563415 (+ 496us) server_negotiation.cc:407] Connection header received
0618 22:27:54.563596 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:54.563599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:54.563659 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:54.563745 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:54.565630 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.566353 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.567709 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.568117 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.568202 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:54.571266 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:27:54.571275 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:27:54.571292 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:27:54.571335 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:27:54.574370 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:54.575051 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:54.575060 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:54.575070 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:54.575290 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:27:54.575821 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:27:54.575825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:27:54.575827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:27:54.576269 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:27:54.576578 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0618 22:27:54.576984 (+ 406us) server_negotiation.cc:299] Negotiation successful
0618 22:27:54.577343 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":855,"thread_start_us":751,"threads_started":1}
Jun 18 22:27:54 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:54 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:54.584830 20426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:54.577673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.195:43431)
0618 22:27:54.577848 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:54.577867 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:54.577953 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:54.578267 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:54.578270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:54.578556 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:54.578690 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:54.578698 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.579654 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.579659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:54.580726 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:54.580741 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.581516 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.581520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:54.581639 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:54.582108 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:54.582125 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:54.584639 (+ 2514us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260618 22:27:54.585047 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:54.577741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:43431 (local address 127.10.242.254:45263)
0618 22:27:54.577895 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:54.577900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:54.577915 (+ 15us) server_negotiation.cc:407] Connection header received
0618 22:27:54.578049 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:54.578054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:54.578141 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:54.578242 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:54.578808 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.579523 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.580873 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.581401 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.581497 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:54.584803 (+ 3306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.195:43431: BlockingRecv error: recv got EOF from 127.10.242.195:43431 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 18 22:27:54 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:54 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:54.592370 20426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:54.585470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.195:37261)
0618 22:27:54.585627 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:54.585646 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:54.585741 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:54.585992 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:54.585995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:54.586238 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:54.586371 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:54.586378 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.587265 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.587269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:54.588145 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:54.588156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.588786 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.588791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:54.588915 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:54.589431 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:54.589448 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:54.592096 (+ 2648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260618 22:27:54.592581 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:54.585556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:37261 (local address 127.10.242.254:45263)
0618 22:27:54.585783 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:54.585787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:54.585800 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:27:54.585836 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:54.585839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:54.585885 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:54.585963 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:54.586473 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.587153 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.588245 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.588671 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.588753 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:54.592248 (+ 3495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.195:37261: BlockingRecv error: recv got EOF from 127.10.242.195:37261 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
W20260618 22:27:54.592748 20424 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 18 22:27:54 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:54 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:54.719056 20429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:54.711089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:42225)
0618 22:27:54.711572 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:54.711591 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:54.711728 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:54.712078 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:54.712081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:54.712495 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:54.712674 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:54.712682 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.713971 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.713975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:54.714910 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:54.714922 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.715563 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.715568 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:54.715688 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:54.716221 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:54.716237 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:54.718823 (+ 2586us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":248,"threads_started":1}
W20260618 22:27:54.719272 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:54.711158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:42225 (local address 127.10.242.254:45263)
0618 22:27:54.711313 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:54.711317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:54.711673 (+ 356us) server_negotiation.cc:407] Connection header received
0618 22:27:54.711854 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:54.711858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:54.711920 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:54.712032 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:54.712796 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.713846 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.715033 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:54.715425 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:54.715521 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:54.719070 (+ 3549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.193:42225: BlockingRecv error: recv got EOF from 127.10.242.193:42225 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 18 22:27:55 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:55 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:55.156109 20430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:55.147297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:57883)
0618 22:27:55.147791 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:55.147815 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:55.147967 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:55.148375 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:55.148380 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:55.148938 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:55.149215 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:55.149229 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:55.150504 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:55.150510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:55.151788 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:55.151802 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:55.152501 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:55.152507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:55.152672 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:55.153267 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:55.153283 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:55.155883 (+ 2600us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":272,"threads_started":1}
W20260618 22:27:55.156292 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:55.147366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:57883 (local address 127.10.242.254:45263)
0618 22:27:55.147582 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:55.147589 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:55.147905 (+ 316us) server_negotiation.cc:407] Connection header received
0618 22:27:55.148136 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:55.148140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:55.148227 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:55.148322 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:55.149350 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:55.150370 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:55.151919 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:55.152324 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:55.152437 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:55.156069 (+ 3632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.194:57883: BlockingRecv error: recv got EOF from 127.10.242.194:57883 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 18 22:27:55 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:55 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:55.601824 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:55.593438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.195:39199)
0618 22:27:55.593945 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:55.593961 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:55.594124 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:55.594455 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:55.594460 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:55.594900 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:55.595090 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:55.595099 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:55.596084 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:55.596090 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:55.597016 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:55.597027 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:55.597898 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:55.597905 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:55.598066 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:55.598675 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:55.598692 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:55.601599 (+ 2907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":288,"threads_started":1}
W20260618 22:27:55.601996 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:55.593539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:39199 (local address 127.10.242.254:45263)
0618 22:27:55.593737 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:55.593743 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:55.594077 (+ 334us) server_negotiation.cc:407] Connection header received
0618 22:27:55.594249 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:55.594253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:55.594313 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:55.594395 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:55.595241 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:55.595954 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:55.597156 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:55.597717 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:55.597834 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:55.601785 (+ 3951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.195:39199: BlockingRecv error: recv got EOF from 127.10.242.195:39199 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 18 22:27:55 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:55 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:55.727754 20432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:55.719999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:56257)
0618 22:27:55.720508 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:55.720529 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:55.720679 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:55.721072 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:55.721111 (+ 39us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:55.721484 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:55.721648 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:55.721657 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:55.722556 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:55.722560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:55.723456 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:55.723466 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:55.724058 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:55.724062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:55.724207 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:55.724741 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:55.724758 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:55.727433 (+ 2675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":253,"threads_started":1}
W20260618 22:27:55.728018 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:55.720046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:56257 (local address 127.10.242.254:45263)
0618 22:27:55.720281 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:55.720287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:55.720609 (+ 322us) server_negotiation.cc:407] Connection header received
0618 22:27:55.720817 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:55.720822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:55.720898 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:55.720998 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:55.721758 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:55.722444 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:55.723562 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:55.723928 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:55.724014 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:55.727682 (+ 3668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.193:56257: BlockingRecv error: recv got EOF from 127.10.242.193:56257 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 18 22:27:56 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:56 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:56.171300 20433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:56.157045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:49627)
0618 22:27:56.157590 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:56.157610 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:56.157743 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:56.158118 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:56.158123 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:56.158516 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:56.158683 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:56.158692 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:56.159713 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:56.159717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:56.160608 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:56.160622 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:56.161372 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:56.161378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:56.161526 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:56.168266 (+ 6740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:56.168288 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:56.171058 (+ 2770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":281,"threads_started":1}
W20260618 22:27:56.171620 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:56.157188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:49627 (local address 127.10.242.254:45263)
0618 22:27:56.157398 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:56.157405 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:56.157690 (+ 285us) server_negotiation.cc:407] Connection header received
0618 22:27:56.157882 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:56.157886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:56.157962 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:56.158050 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:56.158844 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:56.159578 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:56.160741 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:56.161210 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:56.161334 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:56.171336 (+ 10002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.194:49627: BlockingRecv error: recv got EOF from 127.10.242.194:49627 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 18 22:27:56 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:56 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:56.610889 20434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:56.602674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.195:55781)
0618 22:27:56.603198 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:56.603213 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:56.603349 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:56.603675 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:56.603680 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:56.604118 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:56.604282 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:56.604291 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:56.605333 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:56.605340 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:56.606228 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:56.606241 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:56.606923 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:56.606929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:56.607096 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:56.607660 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:56.607676 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:56.610568 (+ 2892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":294,"threads_started":1}
W20260618 22:27:56.611056 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:56.602844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:55781 (local address 127.10.242.254:45263)
0618 22:27:56.603066 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:56.603072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:56.603300 (+ 228us) server_negotiation.cc:407] Connection header received
0618 22:27:56.603474 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:56.603478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:56.603544 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:56.603624 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:56.604426 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:56.605201 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:56.606362 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:56.606775 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:56.606881 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:56.610810 (+ 3929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.195:55781: BlockingRecv error: recv got EOF from 127.10.242.195:55781 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Jun 18 22:27:56 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:56 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:56.736653 20435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:56.728679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:53027)
0618 22:27:56.729217 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:56.729232 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:56.729346 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:56.729623 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:56.729627 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:56.729939 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:56.730123 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:56.730132 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:56.731101 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:56.731105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:56.732017 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:56.732029 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:56.732815 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:56.732819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:56.732935 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:56.733558 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:56.733574 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:56.736374 (+ 2800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":282,"threads_started":1}
W20260618 22:27:56.736783 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:56.728762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:53027 (local address 127.10.242.254:45263)
0618 22:27:56.728967 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:56.728973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:56.729317 (+ 344us) server_negotiation.cc:407] Connection header received
0618 22:27:56.729436 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:56.729440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:56.729501 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:56.729593 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:56.730247 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:56.730987 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:56.732174 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:56.732690 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:56.732816 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:56.736570 (+ 3754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.193:53027: BlockingRecv error: recv got EOF from 127.10.242.193:53027 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Jun 18 22:27:57 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:57 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:57.180526 20436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:57.172172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:59699)
0618 22:27:57.172636 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:57.172657 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:57.172797 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:57.173157 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:57.173162 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:57.173501 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:57.173663 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:57.173673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:57.174693 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:57.174709 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:57.175684 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:57.175700 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:57.176714 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:57.176720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:57.176888 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:57.177595 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:57.177616 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:57.180253 (+ 2637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":242,"threads_started":1}
W20260618 22:27:57.180666 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:57.172317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:59699 (local address 127.10.242.254:45263)
0618 22:27:57.172545 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:57.172551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:57.172765 (+ 214us) server_negotiation.cc:407] Connection header received
0618 22:27:57.172916 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:57.172921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:57.173000 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:57.173109 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:57.173809 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:57.174544 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:57.175926 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:57.176562 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:57.176699 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:57.180438 (+ 3739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.194:59699: BlockingRecv error: recv got EOF from 127.10.242.194:59699 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Jun 18 22:27:57 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:57 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:57.620198 20437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:57.612029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.195:41791)
0618 22:27:57.612461 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:57.612476 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:57.612606 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:57.612908 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:57.612912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:57.613332 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:57.613490 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:57.613499 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:57.614487 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:57.614491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:57.615356 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:57.615367 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:57.616037 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:57.616044 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:57.616200 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:57.616754 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:57.616770 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:57.619905 (+ 3135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":235,"threads_started":1}
W20260618 22:27:57.620282 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:57.612115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:41791 (local address 127.10.242.254:45263)
0618 22:27:57.612296 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:57.612302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:57.612560 (+ 258us) server_negotiation.cc:407] Connection header received
0618 22:27:57.612720 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:57.612726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:57.612786 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:57.612863 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:57.613641 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:57.614352 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:57.615488 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:57.615903 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:57.616022 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:57.620106 (+ 4084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.195:41791: BlockingRecv error: recv got EOF from 127.10.242.195:41791 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 18 22:27:57 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:57 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:57.746116 20438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:57.737707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:48729)
0618 22:27:57.738262 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:57.738278 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:57.738412 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:57.738741 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:57.738746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:57.739121 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:57.739265 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:57.739273 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:57.740397 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:57.740401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:57.741651 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:57.741663 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:57.742355 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:57.742360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:57.742490 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:57.743035 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:57.743051 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:57.745867 (+ 2816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":346,"threads_started":1}
W20260618 22:27:57.746325 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:57.737826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:48729 (local address 127.10.242.254:45263)
0618 22:27:57.738045 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:57.738051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:57.738366 (+ 315us) server_negotiation.cc:407] Connection header received
0618 22:27:57.738543 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:57.738547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:57.738609 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:57.738689 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:57.739399 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:57.740263 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:57.741772 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:57.742205 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:57.742309 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:57.746097 (+ 3788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.193:48729: BlockingRecv error: recv got EOF from 127.10.242.193:48729 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Jun 18 22:27:58 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:58 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:58.189651 20439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:58.181456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:51245)
0618 22:27:58.181927 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:58.181946 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:58.182062 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:58.182341 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:58.182344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:58.182676 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:58.182879 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:58.182889 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:58.183926 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:58.183930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:58.184933 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:58.184945 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:58.185616 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:58.185619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:58.185740 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:58.186320 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:58.186338 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:58.189319 (+ 2981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":256,"threads_started":1}
W20260618 22:27:58.189738 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:58.181554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:51245 (local address 127.10.242.254:45263)
0618 22:27:58.181709 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:58.181713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:58.182068 (+ 355us) server_negotiation.cc:407] Connection header received
0618 22:27:58.182148 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:58.182152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:58.182228 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:58.182309 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:58.183043 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:58.183739 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:58.185060 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:58.185502 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:58.185600 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:58.189511 (+ 3911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.194:51245: BlockingRecv error: recv got EOF from 127.10.242.194:51245 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Jun 18 22:27:58 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:58 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:58.629254 20440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:58.621193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.195:34613)
0618 22:27:58.621672 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:58.621686 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:58.621814 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:58.622153 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:58.622157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:58.622538 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:58.622677 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:58.622685 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:58.623623 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:58.623629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:58.624597 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:58.624609 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:58.625333 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:58.625339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:58.625504 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:58.626058 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:58.626075 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:58.628900 (+ 2825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":269,"threads_started":1}
W20260618 22:27:58.629549 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:58.621315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:34613 (local address 127.10.242.254:45263)
0618 22:27:58.621535 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:58.621542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:58.621768 (+ 226us) server_negotiation.cc:407] Connection header received
0618 22:27:58.621932 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:58.621938 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:58.622029 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:58.622112 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:58.622812 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:58.623504 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:58.624729 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:58.625195 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:58.625308 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:58.629108 (+ 3800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.195:34613: BlockingRecv error: recv got EOF from 127.10.242.195:34613 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 18 22:27:58 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:58 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:58.755342 20441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:58.747081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:60445)
0618 22:27:58.747560 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:58.747576 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:58.747700 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:58.748021 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:58.748026 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:58.748475 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:58.748672 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:58.748683 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:58.749764 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:58.749769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:58.750790 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:58.750805 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:58.751654 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:58.751658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:58.751820 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:58.752358 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:58.752383 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:58.755116 (+ 2733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":250,"threads_started":1}
W20260618 22:27:58.755543 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:58.747167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:60445 (local address 127.10.242.254:45263)
0618 22:27:58.747389 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:58.747396 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:58.747658 (+ 262us) server_negotiation.cc:407] Connection header received
0618 22:27:58.747815 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:58.747821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:58.747892 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:58.747974 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:58.748801 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:58.749641 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:58.750920 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:58.751494 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:58.751619 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:58.755326 (+ 3707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.193:60445: BlockingRecv error: recv got EOF from 127.10.242.193:60445 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Jun 18 22:27:59 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:59 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:59.199184 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:59.190728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:46809)
0618 22:27:59.191286 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:59.191307 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:59.191454 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:59.191773 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:59.191776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:59.192136 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:59.192320 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:59.192331 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:59.193601 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:59.193608 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:59.194775 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:59.194787 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:59.195435 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:59.195440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:59.195558 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:59.196147 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:59.196163 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:59.198940 (+ 2777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":285,"threads_started":1}
W20260618 22:27:59.199350 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:59.190892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:46809 (local address 127.10.242.254:45263)
0618 22:27:59.191065 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:59.191071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:59.191397 (+ 326us) server_negotiation.cc:407] Connection header received
0618 22:27:59.191552 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:59.191556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:59.191643 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:59.191736 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:59.192498 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:59.193468 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:59.194913 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:59.195314 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:59.195422 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:59.199124 (+ 3702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.194:46809: BlockingRecv error: recv got EOF from 127.10.242.194:46809 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Jun 18 22:27:59 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:59 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:59.639207 20443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:59.630523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.195:47867)
0618 22:27:59.631025 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:59.631044 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:59.631189 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:59.631513 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:59.631518 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:59.631927 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:59.632083 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:59.632092 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:59.633124 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:59.633130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:59.634046 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:59.634058 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:59.634741 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:59.634746 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:59.634905 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:59.635482 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:59.635589 (+ 107us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:59.638961 (+ 3372us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":218,"threads_started":1}
W20260618 22:27:59.639428 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:59.630720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:47867 (local address 127.10.242.254:45263)
0618 22:27:59.630935 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:59.630942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:59.631184 (+ 242us) server_negotiation.cc:407] Connection header received
0618 22:27:59.631305 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:59.631311 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:59.631382 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:59.631462 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:59.632239 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:59.632970 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:59.634178 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:59.634599 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:59.634708 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:59.639198 (+ 4490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.195:47867: BlockingRecv error: recv got EOF from 127.10.242.195:47867 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Jun 18 22:27:59 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:27:59 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:27:59.764191 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:27:59.756273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:40985)
0618 22:27:59.756860 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0618 22:27:59.756878 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:27:59.756995 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:27:59.757277 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:27:59.757280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:27:59.757655 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:27:59.757802 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:59.757810 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:59.758827 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:59.758831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:59.759727 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:27:59.759740 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:27:59.760649 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:27:59.760655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:27:59.760789 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:59.761387 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:27:59.761410 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:27:59.763983 (+ 2573us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":326,"threads_started":1}
W20260618 22:27:59.764374 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:27:59.756435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:40985 (local address 127.10.242.254:45263)
0618 22:27:59.756631 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0618 22:27:59.756635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:27:59.756963 (+ 328us) server_negotiation.cc:407] Connection header received
0618 22:27:59.757093 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:27:59.757097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:27:59.757158 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:27:59.757239 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:27:59.757932 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:59.758643 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:59.759862 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:27:59.760466 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:27:59.760587 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:27:59.764164 (+ 3577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.193:40985: BlockingRecv error: recv got EOF from 127.10.242.193:40985 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Jun 18 22:28:00 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:00 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:00.208788 20445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:00.200114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:59665)
0618 22:28:00.200655 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:00.200696 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:00.200836 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:00.201276 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:00.201282 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:00.201702 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:00.201894 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:00.201907 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:00.202920 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:00.202926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:00.204053 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:00.204070 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:00.204830 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:00.204837 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:00.205024 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:00.205647 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:00.205665 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:00.208444 (+ 2779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":305,"threads_started":1}
W20260618 22:28:00.208936 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:00.200252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:59665 (local address 127.10.242.254:45263)
0618 22:28:00.200493 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:00.200500 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:00.200780 (+ 280us) server_negotiation.cc:407] Connection header received
0618 22:28:00.201008 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:00.201012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:00.201131 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:00.201225 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:00.202042 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:00.202776 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:00.204204 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:00.204662 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:00.204779 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:00.208692 (+ 3913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.194:59665: BlockingRecv error: recv got EOF from 127.10.242.194:59665 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Jun 18 22:28:00 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:00 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:00.648399 20446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:00.640161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.195:52959)
0618 22:28:00.640642 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:00.640658 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:00.640770 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:00.641329 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:00.641333 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:00.641705 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:00.641864 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:00.641873 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:00.643011 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:00.643016 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:00.643930 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:00.643949 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:00.644701 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:00.644707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:00.644839 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:00.645503 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:00.645522 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:00.648151 (+ 2629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":252,"threads_started":1}
W20260618 22:28:00.648712 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:00.640215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:52959 (local address 127.10.242.254:45263)
0618 22:28:00.640392 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:00.640397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:00.640758 (+ 361us) server_negotiation.cc:407] Connection header received
0618 22:28:00.640933 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:00.640937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:00.641016 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:00.641145 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:00.642063 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:00.642796 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:00.644075 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:00.644531 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:00.644632 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:00.648392 (+ 3760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.195:52959: BlockingRecv error: recv got EOF from 127.10.242.195:52959 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Jun 18 22:28:00 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:00 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:00.773126 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:00.765140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:45787)
0618 22:28:00.765659 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:00.765674 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:00.765798 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:00.766087 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:00.766091 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:00.766532 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:00.766689 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:00.766698 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:00.767681 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:00.767685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:00.768582 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:00.768596 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:00.769315 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:00.769321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:00.769470 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:00.770179 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:00.770196 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:00.772767 (+ 2571us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":222,"threads_started":1}
W20260618 22:28:00.773216 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:00.765191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:45787 (local address 127.10.242.254:45263)
0618 22:28:00.765375 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:00.765382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:00.765757 (+ 375us) server_negotiation.cc:407] Connection header received
0618 22:28:00.765905 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:00.765909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:00.765966 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:00.766044 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:00.766824 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:00.767544 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:00.768718 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:00.769158 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:00.769259 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:00.772980 (+ 3721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.193:45787: BlockingRecv error: recv got EOF from 127.10.242.193:45787 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Jun 18 22:28:01 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:01 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:01.223232 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:01.209923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:47109)
0618 22:28:01.210518 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:01.210537 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:01.210697 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:01.211182 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:01.211187 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:01.211673 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:01.211860 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:01.211870 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:01.212864 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:01.212870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:01.213790 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:01.213802 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:01.214504 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:01.214510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:01.214668 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:01.219800 (+ 5132us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:01.219822 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:01.222906 (+ 3084us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":281,"threads_started":1}
W20260618 22:28:01.223409 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:01.209956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:47109 (local address 127.10.242.254:45263)
0618 22:28:01.210220 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:01.210226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:01.210619 (+ 393us) server_negotiation.cc:407] Connection header received
0618 22:28:01.210849 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:01.210855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:01.210965 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:01.211129 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:01.212004 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:01.212723 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:01.213928 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:01.214359 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:01.214465 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:01.223095 (+ 8630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.194:47109: BlockingRecv error: recv got EOF from 127.10.242.194:47109 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
Jun 18 22:28:01 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:01 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:01.657683 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:01.649398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.195:53665)
0618 22:28:01.650127 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:01.650147 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:01.650271 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:01.650647 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:01.650653 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:01.651029 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:01.651198 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:01.651208 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:01.652350 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:01.652357 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:01.653335 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:01.653348 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:01.654102 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:01.654107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:01.654241 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:01.654776 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:01.654793 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:01.657422 (+ 2629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":360,"threads_started":1}
W20260618 22:28:01.657907 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:01.649511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:53665 (local address 127.10.242.254:45263)
0618 22:28:01.649760 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:01.649767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:01.650223 (+ 456us) server_negotiation.cc:407] Connection header received
0618 22:28:01.650384 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:01.650389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:01.650468 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:01.650559 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:01.651326 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:01.652201 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:01.653501 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:01.653975 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:01.654078 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:01.657645 (+ 3567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.195:53665: BlockingRecv error: recv got EOF from 127.10.242.195:53665 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Jun 18 22:28:01 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:01 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:01.782456 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:01.774178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:33823)
0618 22:28:01.774747 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:01.774769 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:01.774949 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:01.775328 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:01.775333 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:01.775800 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:01.776033 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:01.776043 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:01.777043 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:01.777047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:01.778027 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:01.778039 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:01.778774 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:01.778779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:01.778892 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:01.779534 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:01.779551 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:01.782215 (+ 2664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":291,"threads_started":1}
W20260618 22:28:01.782757 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:01.774382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:33823 (local address 127.10.242.254:45263)
0618 22:28:01.774549 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:01.774553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:01.774906 (+ 353us) server_negotiation.cc:407] Connection header received
0618 22:28:01.775066 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:01.775071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:01.775145 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:01.775256 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:01.776164 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:01.776902 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:01.778181 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:01.778601 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:01.778701 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:01.782488 (+ 3787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.193:33823: BlockingRecv error: recv got EOF from 127.10.242.193:33823 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Jun 18 22:28:02 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:02 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:02.232425 20451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:02.224263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:40681)
0618 22:28:02.224761 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:02.224782 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:02.224891 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:02.225281 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:02.225286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:02.225676 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:02.225826 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:02.225835 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:02.227055 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:02.227060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:02.227969 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:02.227980 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:02.228704 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:02.228709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:02.228837 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:02.229408 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:02.229424 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:02.232188 (+ 2764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":255,"threads_started":1}
W20260618 22:28:02.232605 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:02.224328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:40681 (local address 127.10.242.254:45263)
0618 22:28:02.224527 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:02.224534 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:02.224872 (+ 338us) server_negotiation.cc:407] Connection header received
0618 22:28:02.225001 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:02.225006 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:02.225098 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:02.225188 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:02.225998 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:02.226917 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:02.228106 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:02.228551 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:02.228641 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:02.232402 (+ 3761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.194:40681: BlockingRecv error: recv got EOF from 127.10.242.194:40681 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 18 22:28:02 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:02 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:02.666119 20452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:02.658612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.195:38139)
0618 22:28:02.659142 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:02.659158 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:02.659264 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:02.659536 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:02.659539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:02.659900 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:02.660062 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:02.660070 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:02.660951 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:02.660954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:02.661878 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:02.661889 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:02.662533 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:02.662538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:02.662649 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:02.663193 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:02.663211 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:02.665851 (+ 2640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":270,"threads_started":1}
W20260618 22:28:02.666330 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:02.658707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:38139 (local address 127.10.242.254:45263)
0618 22:28:02.658868 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:02.658873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:02.659221 (+ 348us) server_negotiation.cc:407] Connection header received
0618 22:28:02.659353 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:02.659357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:02.659414 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:02.659490 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:02.660169 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:02.660846 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:02.661981 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:02.662362 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:02.662449 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:02.666070 (+ 3621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.195:38139: BlockingRecv error: recv got EOF from 127.10.242.195:38139 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Jun 18 22:28:02 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:02 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:02.791564 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:02.783360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:40185)
0618 22:28:02.783867 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:02.783885 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:02.784011 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:02.784336 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:02.784341 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:02.784728 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:02.784888 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:02.784898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:02.785965 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:02.785969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:02.787001 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:02.787013 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:02.787802 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:02.787809 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:02.787970 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:02.788495 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:02.788512 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:02.791252 (+ 2740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":284,"threads_started":1}
W20260618 22:28:02.791656 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:02.783446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:40185 (local address 127.10.242.254:45263)
0618 22:28:02.783619 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:02.783626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:02.783964 (+ 338us) server_negotiation.cc:407] Connection header received
0618 22:28:02.784120 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:02.784125 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:02.784203 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:02.784283 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:02.785028 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:02.785821 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:02.787124 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:02.787657 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:02.787767 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:02.791433 (+ 3666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.193:40185: BlockingRecv error: recv got EOF from 127.10.242.193:40185 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Jun 18 22:28:03 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:03 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:03.241876 20454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:03.233549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:51161)
0618 22:28:03.233998 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:03.234018 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:03.234141 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:03.234451 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:03.234454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:03.234843 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:03.235004 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:03.235013 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:03.236173 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:03.236177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:03.237317 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:03.237330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:03.238021 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:03.238027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:03.238240 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:03.238759 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:03.238775 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:03.241554 (+ 2779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":239,"threads_started":1}
W20260618 22:28:03.242024 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:03.233613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:51161 (local address 127.10.242.254:45263)
0618 22:28:03.233833 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:03.233839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:03.234125 (+ 286us) server_negotiation.cc:407] Connection header received
0618 22:28:03.234246 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:03.234250 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:03.234331 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:03.234418 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:03.235157 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:03.236027 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:03.237454 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:03.237877 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:03.238005 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:03.241771 (+ 3766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.194:51161: BlockingRecv error: recv got EOF from 127.10.242.194:51161 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Jun 18 22:28:03 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:03 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:03.675109 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:03.667100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.195:45383)
0618 22:28:03.667638 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:03.667654 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:03.667785 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:03.668120 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:03.668124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:03.668508 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:03.668662 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:03.668671 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:03.669674 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:03.669679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:03.670586 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:03.670596 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:03.671461 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:03.671467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:03.671598 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:03.672187 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:03.672206 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:03.674878 (+ 2672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":289,"threads_started":1}
W20260618 22:28:03.675303 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:03.667211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:45383 (local address 127.10.242.254:45263)
0618 22:28:03.667467 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:03.667473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:03.667741 (+ 268us) server_negotiation.cc:407] Connection header received
0618 22:28:03.667916 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:03.667922 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:03.667990 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:03.668072 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:03.668802 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:03.669539 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:03.670713 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:03.671290 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:03.671410 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:03.675100 (+ 3690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.195:45383: BlockingRecv error: recv got EOF from 127.10.242.195:45383 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Jun 18 22:28:03 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:03 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:03.800488 20456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:03.792503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:49523)
0618 22:28:03.793010 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:03.793026 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:03.793181 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:03.793475 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:03.793479 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:03.793866 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:03.794051 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:03.794060 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:03.795066 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:03.795071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:03.796022 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:03.796034 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:03.796729 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:03.796734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:03.796883 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:03.797478 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:03.797497 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:03.800248 (+ 2751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":279,"threads_started":1}
W20260618 22:28:03.800657 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:03.792644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:49523 (local address 127.10.242.254:45263)
0618 22:28:03.792864 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:03.792870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:03.793112 (+ 242us) server_negotiation.cc:407] Connection header received
0618 22:28:03.793292 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:03.793298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:03.793356 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:03.793433 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:03.794194 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:03.794934 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:03.796156 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:03.796587 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:03.796692 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:03.800426 (+ 3734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.193:49523: BlockingRecv error: recv got EOF from 127.10.242.193:49523 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Jun 18 22:28:04 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:04 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:04.250670 20457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:04.242826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:56783)
0618 22:28:04.243299 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:04.243319 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:04.243428 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:04.243691 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:04.243694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:04.244031 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:04.244172 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:04.244180 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:04.245155 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:04.245159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:04.246092 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:04.246106 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:04.246794 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:04.246799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:04.246914 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:04.247468 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:04.247508 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:04.250417 (+ 2909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":297,"threads_started":1}
W20260618 22:28:04.250833 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:04.242906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:56783 (local address 127.10.242.254:45263)
0618 22:28:04.243114 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:04.243123 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:04.243394 (+ 271us) server_negotiation.cc:407] Connection header received
0618 22:28:04.243525 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:04.243528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:04.243588 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:04.243682 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:04.244277 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:04.244979 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:04.246210 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:04.246639 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:04.246738 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:04.250613 (+ 3875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.194:56783: BlockingRecv error: recv got EOF from 127.10.242.194:56783 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Jun 18 22:28:04 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:04 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:04.684032 20458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:04.675948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.195:38737)
0618 22:28:04.676525 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:04.676562 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:04.676676 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:04.676948 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:04.676951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:04.677332 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:04.677474 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:04.677482 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:04.678586 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:04.678591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:04.679561 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:04.679574 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:04.680343 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:04.680349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:04.680510 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:04.681055 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:04.681075 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:04.683767 (+ 2692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":384,"threads_started":1}
W20260618 22:28:04.684281 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:04.676157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:38737 (local address 127.10.242.254:45263)
0618 22:28:04.676408 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:04.676414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:04.676646 (+ 232us) server_negotiation.cc:407] Connection header received
0618 22:28:04.676776 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:04.676780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:04.676840 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:04.676920 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:04.677671 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:04.678421 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:04.679715 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:04.680194 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:04.680303 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:04.683992 (+ 3689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.195:38737: BlockingRecv error: recv got EOF from 127.10.242.195:38737 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
Jun 18 22:28:04 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:04 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:04.809613 20459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:04.801437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:40685)
0618 22:28:04.801937 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:04.801955 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:04.802081 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:04.802386 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:04.802390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:04.802833 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:04.802998 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:04.803007 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:04.803954 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:04.803960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:04.804824 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:04.804834 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:04.805604 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:04.805610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:04.805774 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:04.806352 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:04.806376 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:04.809250 (+ 2874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":269,"threads_started":1}
W20260618 22:28:04.809752 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:04.801531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:40685 (local address 127.10.242.254:45263)
0618 22:28:04.801698 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:04.801703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:04.802038 (+ 335us) server_negotiation.cc:407] Connection header received
0618 22:28:04.802191 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:04.802196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:04.802260 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:04.802337 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:04.803133 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:04.803834 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:04.804953 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:04.805465 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:04.805584 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:04.809488 (+ 3904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.193:40685: BlockingRecv error: recv got EOF from 127.10.242.193:40685 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 18 22:28:05 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:05 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:05.259899 20460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:05.251519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:53479)
0618 22:28:05.251975 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:05.252000 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:05.252122 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:05.252408 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:05.252412 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:05.252775 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:05.252936 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:05.252945 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:05.254059 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:05.254063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:05.254957 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:05.254969 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:05.255653 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:05.255659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:05.255837 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:05.256568 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:05.256592 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:05.259627 (+ 3035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":251,"threads_started":1}
W20260618 22:28:05.260076 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:05.251650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:53479 (local address 127.10.242.254:45263)
0618 22:28:05.251855 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:05.251861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:05.252069 (+ 208us) server_negotiation.cc:407] Connection header received
0618 22:28:05.252218 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:05.252222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:05.252283 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:05.252360 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:05.253144 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:05.253887 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:05.255090 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:05.255482 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:05.255576 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:05.259831 (+ 4255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.194:53479: BlockingRecv error: recv got EOF from 127.10.242.194:53479 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Jun 18 22:28:05 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:05 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:05.693894 20461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:05.685114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.195:43261)
0618 22:28:05.685631 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:05.685647 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:05.685758 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:05.686177 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:05.686182 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:05.686713 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:05.686917 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:05.686927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:05.688012 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:05.688017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:05.688905 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:05.688918 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:05.689806 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:05.689812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:05.690039 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:05.690645 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:05.690664 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:05.693651 (+ 2987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":283,"threads_started":1}
W20260618 22:28:05.694234 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:05.685205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:43261 (local address 127.10.242.254:45263)
0618 22:28:05.685421 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:05.685427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:05.685711 (+ 284us) server_negotiation.cc:407] Connection header received
0618 22:28:05.685876 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:05.685880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:05.685951 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:05.686058 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:05.687090 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:05.687865 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:05.689128 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:05.689570 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:05.689729 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:05.693920 (+ 4191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.195:43261: BlockingRecv error: recv got EOF from 127.10.242.195:43261 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Jun 18 22:28:05 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:05 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:05.818796 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:05.810593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:33495)
0618 22:28:05.811105 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:05.811123 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:05.811282 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:05.811625 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:05.811630 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:05.812019 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:05.812164 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:05.812172 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:05.813237 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:05.813243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:05.814137 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:05.814148 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:05.814864 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:05.814870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:05.815035 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:05.815574 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:05.815589 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:05.818571 (+ 2982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":295,"threads_started":1}
W20260618 22:28:05.819013 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:05.810679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:33495 (local address 127.10.242.254:45263)
0618 22:28:05.810904 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:05.810909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:05.811205 (+ 296us) server_negotiation.cc:407] Connection header received
0618 22:28:05.811400 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:05.811406 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:05.811473 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:05.811549 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:05.812315 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:05.813117 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:05.814287 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:05.814725 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:05.814829 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:05.818758 (+ 3929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.193:33495: BlockingRecv error: recv got EOF from 127.10.242.193:33495 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 18 22:28:06 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:06 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:06.273283 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:06.260812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:53959)
0618 22:28:06.261396 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:06.261424 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:06.261600 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:06.261970 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:06.261974 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:06.262394 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:06.262550 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:06.262560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:06.263529 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:06.263533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:06.264475 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:06.264488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:06.265199 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:06.265204 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:06.265337 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:06.270170 (+ 4833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:06.270190 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:06.272986 (+ 2796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":355,"threads_started":1}
W20260618 22:28:06.273553 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:06.260916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:53959 (local address 127.10.242.254:45263)
0618 22:28:06.261146 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:06.261154 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:06.261541 (+ 387us) server_negotiation.cc:407] Connection header received
0618 22:28:06.261730 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:06.261733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:06.261792 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:06.261870 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:06.262714 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:06.263391 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:06.264591 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:06.265022 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:06.265139 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:06.273277 (+ 8138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.194:53959: BlockingRecv error: recv got EOF from 127.10.242.194:53959 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 18 22:28:06 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:06 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:06.703001 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:06.694865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.195:55683)
0618 22:28:06.695475 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:06.695504 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:06.695612 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:06.695941 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:06.695945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:06.696337 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:06.696490 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:06.696499 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:06.697546 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:06.697551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:06.698516 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:06.698529 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:06.699184 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:06.699189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:06.699343 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:06.699902 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:06.699919 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:06.702768 (+ 2849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":413,"threads_started":1}
W20260618 22:28:06.703284 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:06.694932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:55683 (local address 127.10.242.254:45263)
0618 22:28:06.695120 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:06.695125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:06.695583 (+ 458us) server_negotiation.cc:407] Connection header received
0618 22:28:06.695714 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:06.695719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:06.695784 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:06.695864 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:06.696651 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:06.697367 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:06.698629 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:06.699042 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:06.699134 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:06.702993 (+ 3859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.195:55683: BlockingRecv error: recv got EOF from 127.10.242.195:55683 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Jun 18 22:28:06 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:06 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:06.827907 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:06.819824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:60281)
0618 22:28:06.820323 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:06.820339 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:06.820466 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:06.820832 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:06.820836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:06.821304 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:06.821453 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:06.821462 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:06.822490 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:06.822494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:06.823373 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:06.823384 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:06.824107 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:06.824113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:06.824255 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:06.824765 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:06.824782 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:06.827599 (+ 2817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":254,"threads_started":1}
W20260618 22:28:06.827996 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:06.819904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:60281 (local address 127.10.242.254:45263)
0618 22:28:06.820092 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:06.820098 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:06.820422 (+ 324us) server_negotiation.cc:407] Connection header received
0618 22:28:06.820595 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:06.820600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:06.820697 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:06.820784 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:06.821602 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:06.822349 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:06.823513 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:06.823921 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:06.824030 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:06.827791 (+ 3761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.193:60281: BlockingRecv error: recv got EOF from 127.10.242.193:60281 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 18 22:28:07 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:07 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:07.282012 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:07.274183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:50181)
0618 22:28:07.274716 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:07.274735 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:07.274858 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:07.275197 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:07.275202 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:07.275597 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:07.275748 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:07.275757 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:07.276739 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:07.276743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:07.277674 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:07.277686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:07.278466 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:07.278471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:07.278623 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:07.279195 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:07.279212 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:07.281710 (+ 2498us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":239,"threads_started":1}
W20260618 22:28:07.282130 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:07.274399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:50181 (local address 127.10.242.254:45263)
0618 22:28:07.274620 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:07.274626 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:07.274807 (+ 181us) server_negotiation.cc:407] Connection header received
0618 22:28:07.274966 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:07.274972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:07.275034 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:07.275180 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:07.275881 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:07.276606 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:07.277814 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:07.278299 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:07.278409 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:07.281905 (+ 3496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.194:50181: BlockingRecv error: recv got EOF from 127.10.242.194:50181 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Jun 18 22:28:07 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:07 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:07.712211 20467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:07.703932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.195:37453)
0618 22:28:07.704614 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:07.704634 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:07.704775 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:07.705196 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:07.705200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:07.705637 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:07.705787 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:07.705795 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:07.706781 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:07.706785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:07.707738 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:07.707749 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:07.708453 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:07.708458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:07.708601 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:07.709194 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:07.709212 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:07.711978 (+ 2766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":399,"threads_started":1}
W20260618 22:28:07.712464 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:07.704019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:37453 (local address 127.10.242.254:45263)
0618 22:28:07.704229 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:07.704234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:07.704727 (+ 493us) server_negotiation.cc:407] Connection header received
0618 22:28:07.704908 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:07.704914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:07.704987 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:07.705067 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:07.705917 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:07.706648 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:07.707855 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:07.708279 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:07.708375 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:07.712206 (+ 3831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.195:37453: BlockingRecv error: recv got EOF from 127.10.242.195:37453 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Jun 18 22:28:07 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:07 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:07.836951 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:07.828950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:54935)
0618 22:28:07.829452 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:07.829469 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:07.829578 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:07.829873 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:07.829877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:07.830226 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:07.830394 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:07.830403 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:07.831467 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:07.831472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:07.832395 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:07.832408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:07.833155 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:07.833163 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:07.833302 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:07.833895 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:07.833912 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:07.836716 (+ 2804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":239,"threads_started":1}
W20260618 22:28:07.837188 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:07.829043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:54935 (local address 127.10.242.254:45263)
0618 22:28:07.829385 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:07.829390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:07.829534 (+ 144us) server_negotiation.cc:407] Connection header received
0618 22:28:07.829682 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:07.829686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:07.829757 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:07.829844 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:07.830556 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:07.831291 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:07.832529 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:07.832950 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:07.833043 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:07.836936 (+ 3893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.193:54935: BlockingRecv error: recv got EOF from 127.10.242.193:54935 (error 108)
Metrics: {"server-negotiator.queue_time_us":226}
Jun 18 22:28:08 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:08 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:08.290658 20469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:08.282983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:43071)
0618 22:28:08.283455 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:08.283475 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:08.283585 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:08.283854 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:08.283858 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:08.284173 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:08.284349 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:08.284357 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:08.285350 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:08.285354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:08.286243 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:08.286254 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:08.286909 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:08.286913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:08.287045 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:08.287626 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:08.287643 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:08.290432 (+ 2789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":223,"threads_started":1}
W20260618 22:28:08.290799 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:08.283076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:43071 (local address 127.10.242.254:45263)
0618 22:28:08.283276 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:08.283283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:08.283576 (+ 293us) server_negotiation.cc:407] Connection header received
0618 22:28:08.283672 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:08.283676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:08.283736 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:08.283813 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:08.284462 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:08.285235 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:08.286370 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:08.286802 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:08.286890 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:08.290624 (+ 3734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.194:43071: BlockingRecv error: recv got EOF from 127.10.242.194:43071 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 18 22:28:08 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:08 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.195@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:08.721263 20470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:08.713159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.195:46335)
0618 22:28:08.713599 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:08.713615 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:08.713731 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:08.714028 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:08.714031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:08.714486 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:08.714669 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:08.714678 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:08.715671 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:08.715676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:08.716614 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:08.716626 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:08.717274 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:08.717279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:08.717407 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:08.718019 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:08.718038 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:08.720909 (+ 2871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":235,"threads_started":1}
W20260618 22:28:08.721391 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:08.713317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:46335 (local address 127.10.242.254:45263)
0618 22:28:08.713503 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:08.713509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:08.713721 (+ 212us) server_negotiation.cc:407] Connection header received
0618 22:28:08.713837 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:08.713842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:08.713906 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:08.714014 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:08.714825 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:08.715518 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:08.716729 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:08.717146 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:08.717234 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:08.721154 (+ 3920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.195:46335: BlockingRecv error: recv got EOF from 127.10.242.195:46335 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Jun 18 22:28:08 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:08 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.193@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
W20260618 22:28:08.846395 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:08.838331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:40053 (local address 127.10.242.254:45263)
0618 22:28:08.838515 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:08.838522 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:08.838913 (+ 391us) server_negotiation.cc:407] Connection header received
0618 22:28:08.839051 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:08.839057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:08.839138 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:08.839220 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:08.839972 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:08.840674 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:08.841889 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:08.842332 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:08.842437 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:08.846225 (+ 3788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.193:40053: BlockingRecv error: recv got EOF from 127.10.242.193:40053 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260618 22:28:08.846412 20471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:08.838295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.193:40053)
0618 22:28:08.838807 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:08.838823 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:08.838949 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:08.839273 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:08.839278 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:08.839651 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:08.839800 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:08.839810 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:08.840806 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:08.840812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:08.841748 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:08.841759 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:08.842467 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:08.842472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:08.842630 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:08.843178 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:08.843197 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:08.846071 (+ 2874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":266,"threads_started":1}
Jun 18 22:28:09 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:28:09 dist-test-slave-0956 krb5kdc[19427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.194@KRBTEST.COM for oryx/127.10.242.254@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:28:09.300146 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:09.291854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45263 (local address 127.10.242.194:43817)
0618 22:28:09.292422 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:09.292444 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:09.292576 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:09.292847 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:09.292850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:09.293287 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:09.293489 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:09.293499 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:09.294574 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:09.294578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:09.295583 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:09.295596 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:09.296265 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:09.296269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:09.296390 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:09.296961 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:09.296977 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:09.299891 (+ 2914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.254:45263: Server oryx/127.10.242.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":311,"threads_started":1}
W20260618 22:28:09.300272 20427 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:09.291958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:43817 (local address 127.10.242.254:45263)
0618 22:28:09.292122 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:09.292126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:09.292514 (+ 388us) server_negotiation.cc:407] Connection header received
0618 22:28:09.292668 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:09.292672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:09.292729 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:09.292806 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:09.293648 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:09.294422 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:09.295716 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:09.296118 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:09.296214 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:09.300057 (+ 3843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.194:43817: BlockingRecv error: recv got EOF from 127.10.242.194:43817 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:28:09.561988 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 20014
I20260618 22:28:09.573987 20474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:09.563049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55894 (local address 127.10.242.193:36723)
0618 22:28:09.563587 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:09.563594 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:09.563620 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:28:09.563677 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:09.563684 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:09.563905 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:09.564049 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:09.565501 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:09.566524 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:09.567760 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:09.568324 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:09.568424 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:09.571064 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:09.571073 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:09.571088 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:28:09.571120 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:09.573487 (+ 2367us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.242.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0618 22:28:09.573757 (+ 270us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55894: Request ticket server kudu/127.10.242.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":267,"threads_started":1}
W20260618 22:28:09.574082 20474 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55894: Request ticket server kudu/127.10.242.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260618 22:28:09.574214 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 20014: Not authorized: Client connection negotiation failed: client connection to 127.10.242.193:36723: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.242.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260618 22:28:09.574395 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 20014
I20260618 22:28:09.587441 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 20165
I20260618 22:28:09.599115 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:09.588404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44726 (local address 127.10.242.194:36271)
0618 22:28:09.588962 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:09.588969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:09.588998 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:28:09.589054 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:09.589061 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:09.589324 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:09.589498 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:09.590739 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:09.591779 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:09.592959 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:09.593555 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:09.593671 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:09.596060 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:09.596069 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:09.596083 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:28:09.596116 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:09.598471 (+ 2355us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.242.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0618 22:28:09.598798 (+ 327us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44726: Request ticket server kudu/127.10.242.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":276,"threads_started":1}
W20260618 22:28:09.599207 20475 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44726: Request ticket server kudu/127.10.242.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260618 22:28:09.599339 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 20165: Not authorized: Client connection negotiation failed: client connection to 127.10.242.194:36271: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.242.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260618 22:28:09.599519 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 20165
I20260618 22:28:09.612449 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 20296
I20260618 22:28:09.624176 20476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:09.613511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44422 (local address 127.10.242.195:39419)
0618 22:28:09.614085 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:09.614092 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:09.614120 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:28:09.614183 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:09.614190 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:09.614421 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:09.614598 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:09.615853 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:09.616976 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:09.618254 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:09.618834 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:09.618943 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:09.621406 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:09.621414 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:09.621428 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:28:09.621473 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:09.623695 (+ 2222us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.242.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0618 22:28:09.623945 (+ 250us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44422: Request ticket server kudu/127.10.242.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":289,"threads_started":1}
W20260618 22:28:09.624269 20476 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44422: Request ticket server kudu/127.10.242.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260618 22:28:09.624327 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 20296: Not authorized: Client connection negotiation failed: client connection to 127.10.242.195:39419: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.242.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260618 22:28:09.624475 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 20296
I20260618 22:28:09.636945 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 19945
I20260618 22:28:09.638383 20007 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:28:09.749310 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 19945
2026-06-18T22:28:09Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (20800 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-18T22:28:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:28:09Z Disabled control of system clock
I20260618 22:28:09.817611 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:43473
--webserver_interface=127.10.242.254
--webserver_port=0
--builtin_ntp_servers=127.10.242.212:44821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:43473
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260618 22:28:10.160735 20481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:28:10.161026 20481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:28:10.161139 20481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:28:10.168428 20481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:28:10.168532 20481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:28:10.168567 20481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:28:10.168619 20481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:28:10.168681 20481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:28:10.179250 20481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:44821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:43473
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:43473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:28:10.180521 20481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:28:10.182250 20481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:28:10.193992 20487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:10.194185 20486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:10.194087 20489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:28:10.194378 20481 server_base.cc:1034] running on GCE node
I20260618 22:28:10.195427 20481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:28:10.196800 20481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:28:10.198010 20481 hybrid_clock.cc:648] HybridClock initialized: now 1781821690197940 us; error 57 us; skew 500 ppm
I20260618 22:28:10.198406 20481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:28:10.200742 20481 webserver.cc:460] Webserver started at http://127.10.242.254:36313/ using document root <none> and password file <none>
I20260618 22:28:10.201488 20481 fs_manager.cc:359] Metadata directory not provided
I20260618 22:28:10.201582 20481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:28:10.201881 20481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:28:10.204684 20481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "ea9b258cc4304a7fa788af9d2609ecae"
format_stamp: "Formatted at 2026-06-18 22:28:10 on dist-test-slave-0956"
server_key: "9df277e717d5a725de711d0b21231d2e"
server_key_iv: "31f7699f0492fec7f3ced35362c4c0c1"
server_key_version: "encryptionkey@0"
I20260618 22:28:10.205433 20481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "ea9b258cc4304a7fa788af9d2609ecae"
format_stamp: "Formatted at 2026-06-18 22:28:10 on dist-test-slave-0956"
server_key: "9df277e717d5a725de711d0b21231d2e"
server_key_iv: "31f7699f0492fec7f3ced35362c4c0c1"
server_key_version: "encryptionkey@0"
I20260618 22:28:10.210230 20481 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260618 22:28:10.213598 20495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:10.214913 20481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:28:10.215070 20481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "ea9b258cc4304a7fa788af9d2609ecae"
format_stamp: "Formatted at 2026-06-18 22:28:10 on dist-test-slave-0956"
server_key: "9df277e717d5a725de711d0b21231d2e"
server_key_iv: "31f7699f0492fec7f3ced35362c4c0c1"
server_key_version: "encryptionkey@0"
I20260618 22:28:10.215267 20481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:28:10.240901 20481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:28:10.241801 20481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:28:10.242077 20481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:28:10.258962 20481 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:43473
I20260618 22:28:10.258966 20546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:43473 every 8 connection(s)
I20260618 22:28:10.260298 20481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:28:10.264046 20547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:28:10.269492 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 20481
I20260618 22:28:10.269704 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:28:10.270296 11211 external_mini_cluster.cc:1428] Setting key b7d85dcd3dff8d0ff45b37210b093704
I20260618 22:28:10.274633 20547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae: Bootstrap starting.
I20260618 22:28:10.277940 20547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae: Neither blocks nor log segments found. Creating new log.
I20260618 22:28:10.278903 20547 log.cc:824] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae: Log is configured to *not* fsync() on all Append() calls
I20260618 22:28:10.281883 20547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae: No bootstrap required, opened a new log
I20260618 22:28:10.283713 20550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:10.272429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55540 (local address 127.10.242.254:43473)
0618 22:28:10.273444 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:10.273455 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:10.273498 (+ 43us) server_negotiation.cc:407] Connection header received
0618 22:28:10.275112 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:10.275135 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:10.275589 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:10.275917 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:10.276655 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:10.277992 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:10.279393 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:10.280095 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:10.280286 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:10.281470 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:10.281477 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:10.281491 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:28:10.282030 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:10.282121 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:28:10.282155 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:10.282271 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:10.282471 (+ 200us) server_negotiation.cc:299] Negotiation successful
0618 22:28:10.282773 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":366,"threads_started":1}
I20260618 22:28:10.289462 20547 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea9b258cc4304a7fa788af9d2609ecae" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43473 } }
I20260618 22:28:10.289824 20547 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:28:10.289916 20547 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea9b258cc4304a7fa788af9d2609ecae, State: Initialized, Role: FOLLOWER
I20260618 22:28:10.290470 20547 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea9b258cc4304a7fa788af9d2609ecae" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43473 } }
I20260618 22:28:10.290683 20547 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:28:10.290812 20547 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:28:10.290959 20547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:28:10.293321 20547 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea9b258cc4304a7fa788af9d2609ecae" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43473 } }
I20260618 22:28:10.293790 20547 leader_election.cc:302] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea9b258cc4304a7fa788af9d2609ecae; no voters:
I20260618 22:28:10.294312 20547 leader_election.cc:288] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:28:10.294453 20552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:28:10.295049 20552 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae [term 1 LEADER]: Becoming Leader. State: Replica: ea9b258cc4304a7fa788af9d2609ecae, State: Running, Role: LEADER
I20260618 22:28:10.295588 20552 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea9b258cc4304a7fa788af9d2609ecae" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43473 } }
I20260618 22:28:10.296185 20547 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:28:10.298239 20554 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea9b258cc4304a7fa788af9d2609ecae. Latest consensus state: current_term: 1 leader_uuid: "ea9b258cc4304a7fa788af9d2609ecae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea9b258cc4304a7fa788af9d2609ecae" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43473 } } }
I20260618 22:28:10.298679 20554 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae [sys.catalog]: This master's current role is: LEADER
I20260618 22:28:10.300261 20553 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea9b258cc4304a7fa788af9d2609ecae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea9b258cc4304a7fa788af9d2609ecae" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43473 } } }
I20260618 22:28:10.300696 20553 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae [sys.catalog]: This master's current role is: LEADER
I20260618 22:28:10.301714 20561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:28:10.306197 20561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:28:10.314215 20561 catalog_manager.cc:1347] Generated new cluster ID: 81123c574a3d4e7ba35ac4469b3e8dc6
I20260618 22:28:10.314339 20561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:28:10.326447 20561 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:28:10.328248 20561 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:28:10.337847 20561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea9b258cc4304a7fa788af9d2609ecae: Generated new TSK 0
I20260618 22:28:10.338929 20561 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:28:10.348502 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:43473
--builtin_ntp_servers=127.10.242.212:44821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260618 22:28:10.663831 20571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:28:10.664121 20571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:28:10.664232 20571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:28:10.671481 20571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:28:10.671582 20571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:28:10.671729 20571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:28:10.680826 20571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:44821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:43473
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:28:10.682111 20571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:28:10.683832 20571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:28:10.697074 20576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:10.697103 20577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:10.697120 20579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:28:10.698450 20571 server_base.cc:1034] running on GCE node
I20260618 22:28:10.699056 20571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:28:10.700002 20571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:28:10.701169 20571 hybrid_clock.cc:648] HybridClock initialized: now 1781821690701115 us; error 46 us; skew 500 ppm
I20260618 22:28:10.701510 20571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:28:10.703792 20571 webserver.cc:460] Webserver started at http://127.10.242.193:36983/ using document root <none> and password file <none>
I20260618 22:28:10.704493 20571 fs_manager.cc:359] Metadata directory not provided
I20260618 22:28:10.704582 20571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:28:10.704890 20571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:28:10.707820 20571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "5b02a54fa6b440b0abe1654dc204cd3a"
format_stamp: "Formatted at 2026-06-18 22:28:10 on dist-test-slave-0956"
server_key: "e225e3318f31d41e2631cc1b7478bbab"
server_key_iv: "247b34c14c25fc919aec6611f0bbb196"
server_key_version: "encryptionkey@0"
I20260618 22:28:10.708617 20571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "5b02a54fa6b440b0abe1654dc204cd3a"
format_stamp: "Formatted at 2026-06-18 22:28:10 on dist-test-slave-0956"
server_key: "e225e3318f31d41e2631cc1b7478bbab"
server_key_iv: "247b34c14c25fc919aec6611f0bbb196"
server_key_version: "encryptionkey@0"
I20260618 22:28:10.713924 20571 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260618 22:28:10.717497 20585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:10.719017 20571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260618 22:28:10.719210 20571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "5b02a54fa6b440b0abe1654dc204cd3a"
format_stamp: "Formatted at 2026-06-18 22:28:10 on dist-test-slave-0956"
server_key: "e225e3318f31d41e2631cc1b7478bbab"
server_key_iv: "247b34c14c25fc919aec6611f0bbb196"
server_key_version: "encryptionkey@0"
I20260618 22:28:10.719465 20571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:28:10.732676 20571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:28:10.733615 20571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:28:10.733906 20571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:28:10.734820 20571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:28:10.736217 20571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:28:10.736320 20571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:10.736428 20571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:28:10.736496 20571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:10.766322 20571 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:35501
I20260618 22:28:10.766377 20697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:35501 every 8 connection(s)
I20260618 22:28:10.767628 20571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:28:10.768934 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 20571
I20260618 22:28:10.769176 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:28:10.769840 11211 external_mini_cluster.cc:1428] Setting key c80fc91ba51bfe340c1be6315e529181
I20260618 22:28:10.773022 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:43473
--builtin_ntp_servers=127.10.242.212:44821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260618 22:28:10.782042 20550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:10.770220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:33685 (local address 127.10.242.254:43473)
0618 22:28:10.770438 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:10.770444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:10.771987 (+ 1543us) server_negotiation.cc:407] Connection header received
0618 22:28:10.774398 (+ 2411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:10.774402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:10.774469 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:10.774555 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:10.775947 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:10.776719 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:10.778868 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:10.779294 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:10.779396 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:10.780926 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:10.780932 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:10.780936 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:28:10.780960 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:10.781003 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:28:10.781019 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:10.781140 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:10.781633 (+ 493us) server_negotiation.cc:299] Negotiation successful
0618 22:28:10.781732 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260618 22:28:10.783097 20700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:10.770715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:43473 (local address 127.10.242.193:33685)
0618 22:28:10.771838 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:10.771876 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:10.774069 (+ 2193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:10.774825 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:10.774837 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:10.774965 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:10.775754 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:10.775777 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:10.776878 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:10.776883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:10.778693 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:10.778709 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:10.779517 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:10.779523 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:10.779813 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:10.780587 (+ 774us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:28:10.780590 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:10.780632 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:28:10.780717 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:28:10.781265 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:28:10.781275 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:28:10.781282 (+ 7us) client_negotiation.cc:769] Sending connection context
0618 22:28:10.781566 (+ 284us) client_negotiation.cc:240] Negotiation successful
0618 22:28:10.781919 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":377,"threads_started":1}
I20260618 22:28:10.784941 20698 heartbeater.cc:347] Connected to a master server at 127.10.242.254:43473
I20260618 22:28:10.785280 20698 heartbeater.cc:464] Registering TS with master...
I20260618 22:28:10.786053 20698 heartbeater.cc:511] Master 127.10.242.254:43473 requested a full tablet report, sending...
I20260618 22:28:10.788084 20512 ts_manager.cc:194] Registered new tserver with Master: 5b02a54fa6b440b0abe1654dc204cd3a (127.10.242.193:35501)
I20260618 22:28:10.789757 20512 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.193:33685
W20260618 22:28:11.097688 20701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:28:11.098002 20701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:28:11.098129 20701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:28:11.105669 20701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:28:11.105799 20701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:28:11.105947 20701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:28:11.115118 20701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:44821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:43473
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:28:11.116411 20701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:28:11.118193 20701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:28:11.131075 20707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:11.131131 20706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:11.131145 20709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:28:11.131672 20701 server_base.cc:1034] running on GCE node
I20260618 22:28:11.132395 20701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:28:11.133409 20701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:28:11.134618 20701 hybrid_clock.cc:648] HybridClock initialized: now 1781821691134562 us; error 43 us; skew 500 ppm
I20260618 22:28:11.134945 20701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:28:11.137238 20701 webserver.cc:460] Webserver started at http://127.10.242.194:36085/ using document root <none> and password file <none>
I20260618 22:28:11.137943 20701 fs_manager.cc:359] Metadata directory not provided
I20260618 22:28:11.138046 20701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:28:11.138394 20701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:28:11.141264 20701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "3bba5f17774d4bb4b562a5cd88b1c94b"
format_stamp: "Formatted at 2026-06-18 22:28:11 on dist-test-slave-0956"
server_key: "50a42f0d95d095ecc00c1462bd6c56c0"
server_key_iv: "a3d801011a478b306facbe4c29383c09"
server_key_version: "encryptionkey@0"
I20260618 22:28:11.142030 20701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "3bba5f17774d4bb4b562a5cd88b1c94b"
format_stamp: "Formatted at 2026-06-18 22:28:11 on dist-test-slave-0956"
server_key: "50a42f0d95d095ecc00c1462bd6c56c0"
server_key_iv: "a3d801011a478b306facbe4c29383c09"
server_key_version: "encryptionkey@0"
I20260618 22:28:11.147140 20701 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:28:11.150619 20715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:11.151919 20701 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:28:11.152077 20701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "3bba5f17774d4bb4b562a5cd88b1c94b"
format_stamp: "Formatted at 2026-06-18 22:28:11 on dist-test-slave-0956"
server_key: "50a42f0d95d095ecc00c1462bd6c56c0"
server_key_iv: "a3d801011a478b306facbe4c29383c09"
server_key_version: "encryptionkey@0"
I20260618 22:28:11.152271 20701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:28:11.179289 20701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:28:11.180171 20701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:28:11.180469 20701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:28:11.181454 20701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:28:11.182889 20701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:28:11.182983 20701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:11.183094 20701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:28:11.183166 20701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:11.213600 20701 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:43763
I20260618 22:28:11.213680 20827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:43763 every 8 connection(s)
I20260618 22:28:11.214844 20701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:28:11.218302 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 20701
I20260618 22:28:11.218477 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:28:11.219002 11211 external_mini_cluster.cc:1428] Setting key 7a8e0527bffabfc6ea263e4897467cea
I20260618 22:28:11.221454 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:43473
--builtin_ntp_servers=127.10.242.212:44821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260618 22:28:11.227029 20550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:11.217199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:35979 (local address 127.10.242.254:43473)
0618 22:28:11.217385 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:11.217389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:11.218782 (+ 1393us) server_negotiation.cc:407] Connection header received
0618 22:28:11.220857 (+ 2075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:11.220862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:11.220937 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:11.221018 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:11.222180 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:11.222905 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:11.224511 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:11.224898 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:11.225004 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:11.226288 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:11.226293 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:11.226296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:28:11.226314 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:11.226342 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:28:11.226353 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:11.226424 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:11.226636 (+ 212us) server_negotiation.cc:299] Negotiation successful
0618 22:28:11.226726 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260618 22:28:11.228186 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:11.217726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:43473 (local address 127.10.242.194:35979)
0618 22:28:11.218646 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:11.218682 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:11.220605 (+ 1923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:11.221277 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:11.221285 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:11.221391 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:11.222021 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:11.222039 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:11.223047 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:11.223051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:11.224385 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:11.224397 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:11.225137 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:11.225146 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:11.225365 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:11.225996 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:28:11.225999 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:11.226038 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:28:11.226119 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:28:11.226437 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:28:11.226448 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:28:11.226456 (+ 8us) client_negotiation.cc:769] Sending connection context
0618 22:28:11.226605 (+ 149us) client_negotiation.cc:240] Negotiation successful
0618 22:28:11.226978 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":260,"threads_started":1}
I20260618 22:28:11.229671 20828 heartbeater.cc:347] Connected to a master server at 127.10.242.254:43473
I20260618 22:28:11.229952 20828 heartbeater.cc:464] Registering TS with master...
I20260618 22:28:11.230679 20828 heartbeater.cc:511] Master 127.10.242.254:43473 requested a full tablet report, sending...
I20260618 22:28:11.232007 20512 ts_manager.cc:194] Registered new tserver with Master: 3bba5f17774d4bb4b562a5cd88b1c94b (127.10.242.194:43763)
I20260618 22:28:11.233002 20512 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.194:35979
W20260618 22:28:11.538883 20831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:28:11.539175 20831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:28:11.539296 20831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:28:11.546518 20831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:28:11.546622 20831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:28:11.546763 20831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:28:11.555968 20831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:44821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:43473
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:28:11.557312 20831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:28:11.559055 20831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:28:11.571544 20837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:11.571573 20836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:11.571662 20839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:28:11.572256 20831 server_base.cc:1034] running on GCE node
I20260618 22:28:11.572829 20831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:28:11.573779 20831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:28:11.574977 20831 hybrid_clock.cc:648] HybridClock initialized: now 1781821691574930 us; error 42 us; skew 500 ppm
I20260618 22:28:11.575289 20831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:28:11.583158 20831 webserver.cc:460] Webserver started at http://127.10.242.195:40213/ using document root <none> and password file <none>
I20260618 22:28:11.583860 20831 fs_manager.cc:359] Metadata directory not provided
I20260618 22:28:11.583957 20831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:28:11.584271 20831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:28:11.587031 20831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "61569ea1d79f47208fe1e685515f38ce"
format_stamp: "Formatted at 2026-06-18 22:28:11 on dist-test-slave-0956"
server_key: "1535d4d7f46c5baf1e3f9b6f7bde4ea5"
server_key_iv: "690a3496cfb5563af88ba869e8c4bed3"
server_key_version: "encryptionkey@0"
I20260618 22:28:11.587759 20831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "61569ea1d79f47208fe1e685515f38ce"
format_stamp: "Formatted at 2026-06-18 22:28:11 on dist-test-slave-0956"
server_key: "1535d4d7f46c5baf1e3f9b6f7bde4ea5"
server_key_iv: "690a3496cfb5563af88ba869e8c4bed3"
server_key_version: "encryptionkey@0"
I20260618 22:28:11.592773 20831 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260618 22:28:11.596060 20845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:11.597446 20831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:28:11.597589 20831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "61569ea1d79f47208fe1e685515f38ce"
format_stamp: "Formatted at 2026-06-18 22:28:11 on dist-test-slave-0956"
server_key: "1535d4d7f46c5baf1e3f9b6f7bde4ea5"
server_key_iv: "690a3496cfb5563af88ba869e8c4bed3"
server_key_version: "encryptionkey@0"
I20260618 22:28:11.597782 20831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:28:11.616739 20831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:28:11.617645 20831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:28:11.617933 20831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:28:11.618875 20831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:28:11.620370 20831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:28:11.620460 20831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:11.620560 20831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:28:11.620625 20831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:11.651242 20831 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:42095
I20260618 22:28:11.651295 20957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:42095 every 8 connection(s)
I20260618 22:28:11.652542 20831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:28:11.653517 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 20831
I20260618 22:28:11.653707 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:28:11.654090 11211 external_mini_cluster.cc:1428] Setting key 3f1ffefdde4671853415b14551f4648f
I20260618 22:28:11.664304 20550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:11.655187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:42195 (local address 127.10.242.254:43473)
0618 22:28:11.655354 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:11.655358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:11.657110 (+ 1752us) server_negotiation.cc:407] Connection header received
0618 22:28:11.658766 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:11.658769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:11.658830 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:11.658905 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:11.659860 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:11.660534 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:11.662046 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:11.662433 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:11.662521 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:11.663691 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:11.663695 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:11.663698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:28:11.663714 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:11.663744 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:28:11.663755 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:11.663820 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:11.663995 (+ 175us) server_negotiation.cc:299] Negotiation successful
0618 22:28:11.664064 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:28:11.665262 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:11.655854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:43473 (local address 127.10.242.195:42195)
0618 22:28:11.656951 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:11.657003 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:11.658550 (+ 1547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:11.659069 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:11.659076 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:11.659163 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:11.659729 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:11.659744 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:11.660640 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:11.660644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:11.661921 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:11.661933 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:11.662597 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:11.662601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:11.662793 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:11.663419 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:28:11.663422 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:11.663462 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:28:11.663537 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:28:11.663832 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:28:11.663839 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:28:11.663844 (+ 5us) client_negotiation.cc:769] Sending connection context
0618 22:28:11.663975 (+ 131us) client_negotiation.cc:240] Negotiation successful
0618 22:28:11.664247 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":307,"threads_started":1}
I20260618 22:28:11.666604 20958 heartbeater.cc:347] Connected to a master server at 127.10.242.254:43473
I20260618 22:28:11.666850 20958 heartbeater.cc:464] Registering TS with master...
I20260618 22:28:11.667706 20958 heartbeater.cc:511] Master 127.10.242.254:43473 requested a full tablet report, sending...
I20260618 22:28:11.669240 20511 ts_manager.cc:194] Registered new tserver with Master: 61569ea1d79f47208fe1e685515f38ce (127.10.242.195:42095)
I20260618 22:28:11.670127 20511 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.195:42195
I20260618 22:28:11.681474 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260618 22:28:11.687273 20550 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:11.685644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55552 (local address 127.10.242.254:43473)
0618 22:28:11.685807 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:11.685813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:11.686269 (+ 456us) server_negotiation.cc:407] Connection header received
0618 22:28:11.686435 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:11.686439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:11.686499 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:11.686578 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:11.687003 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55552: BlockingRecv error: recv got EOF from 127.0.0.1:55552 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:28:11.687855 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 20571
I20260618 22:28:11.695922 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:11.688750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57388 (local address 127.10.242.193:35501)
0618 22:28:11.689248 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:11.689256 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:11.689281 (+ 25us) server_negotiation.cc:407] Connection header received
0618 22:28:11.689340 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:11.689346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:11.689547 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:11.689668 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:11.690029 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:11.691039 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:11.692640 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:11.693347 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:11.693457 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:11.694428 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:11.694436 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:11.694450 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:28:11.695063 (+ 613us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:11.695158 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:28:11.695186 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:11.695278 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:11.695445 (+ 167us) server_negotiation.cc:299] Negotiation successful
0618 22:28:11.695578 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":243,"threads_started":1}
I20260618 22:28:11.697307 20693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:28:11.778884 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 20571
I20260618 22:28:11.791760 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 20701
I20260618 22:28:11.799178 20969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:11.792641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49720 (local address 127.10.242.194:43763)
0618 22:28:11.793187 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:11.793194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:11.793221 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:28:11.793279 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:11.793285 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:11.793526 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:11.793650 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:11.793963 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:11.794950 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:11.796231 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:11.796846 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:11.796962 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:11.797956 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:11.797964 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:11.797977 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:28:11.798537 (+ 560us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:11.798580 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:28:11.798598 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:11.798695 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:11.798792 (+ 97us) server_negotiation.cc:299] Negotiation successful
0618 22:28:11.798900 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":269,"threads_started":1}
I20260618 22:28:11.800418 20823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:28:11.880448 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 20701
I20260618 22:28:11.895376 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 20831
I20260618 22:28:11.903226 20971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:11.896385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50648 (local address 127.10.242.195:42095)
0618 22:28:11.897015 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:11.897022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:11.897050 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:28:11.897140 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:11.897147 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:11.897371 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:11.897498 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:11.897887 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:11.898882 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:11.899989 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:11.900567 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:11.900660 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:11.901763 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:11.901772 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:11.901786 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:28:11.902422 (+ 636us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:11.902481 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:28:11.902501 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:11.902589 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:11.902737 (+ 148us) server_negotiation.cc:299] Negotiation successful
0618 22:28:11.902872 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":321,"threads_started":1}
I20260618 22:28:11.904409 20953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:28:11.984372 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 20831
I20260618 22:28:11.996773 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 20481
I20260618 22:28:11.998045 20542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:28:12.097044 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 20481
2026-06-18T22:28:12Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2343 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-18T22:28:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:28:12Z Disabled control of system clock
I20260618 22:28:12.161204 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:45967
--webserver_interface=127.10.242.254
--webserver_port=0
--builtin_ntp_servers=127.10.242.212:34331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:45967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260618 22:28:12.475231 20977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:28:12.475509 20977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:28:12.475613 20977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:28:12.482892 20977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:28:12.482995 20977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:28:12.483036 20977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:28:12.483114 20977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:28:12.483171 20977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:28:12.492415 20977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:45967
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:45967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:28:12.493703 20977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:28:12.495421 20977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:28:12.506351 20982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:12.506390 20985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:12.506421 20983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:28:12.507202 20977 server_base.cc:1034] running on GCE node
I20260618 22:28:12.507962 20977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:28:12.509352 20977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:28:12.510552 20977 hybrid_clock.cc:648] HybridClock initialized: now 1781821692510505 us; error 38 us; skew 500 ppm
I20260618 22:28:12.510871 20977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:28:12.513074 20977 webserver.cc:460] Webserver started at http://127.10.242.254:41271/ using document root <none> and password file <none>
I20260618 22:28:12.513767 20977 fs_manager.cc:359] Metadata directory not provided
I20260618 22:28:12.513860 20977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:28:12.514153 20977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:28:12.516855 20977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "a324a4f89bc1495e9f8a0afadb38a25f"
format_stamp: "Formatted at 2026-06-18 22:28:12 on dist-test-slave-0956"
server_key: "1532d73bc9bfc3384a88f1a688520c69"
server_key_iv: "17d55ca947e4538a57a04bdf1c6e5f66"
server_key_version: "encryptionkey@0"
I20260618 22:28:12.517613 20977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "a324a4f89bc1495e9f8a0afadb38a25f"
format_stamp: "Formatted at 2026-06-18 22:28:12 on dist-test-slave-0956"
server_key: "1532d73bc9bfc3384a88f1a688520c69"
server_key_iv: "17d55ca947e4538a57a04bdf1c6e5f66"
server_key_version: "encryptionkey@0"
I20260618 22:28:12.522538 20977 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260618 22:28:12.525904 20991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:12.527227 20977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:28:12.527386 20977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "a324a4f89bc1495e9f8a0afadb38a25f"
format_stamp: "Formatted at 2026-06-18 22:28:12 on dist-test-slave-0956"
server_key: "1532d73bc9bfc3384a88f1a688520c69"
server_key_iv: "17d55ca947e4538a57a04bdf1c6e5f66"
server_key_version: "encryptionkey@0"
I20260618 22:28:12.527578 20977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:28:12.533216 20977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:28:12.533969 20977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:28:12.534238 20977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:28:12.551009 20977 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:45967
I20260618 22:28:12.551026 21042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:45967 every 8 connection(s)
I20260618 22:28:12.552321 20977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:28:12.555959 21043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:28:12.561143 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 20977
I20260618 22:28:12.561321 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:28:12.561915 11211 external_mini_cluster.cc:1428] Setting key 3f18fd11e395e91260a2db8ca2782643
I20260618 22:28:12.567157 21043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f: Bootstrap starting.
I20260618 22:28:12.570195 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:12.564070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45498 (local address 127.10.242.254:45967)
0618 22:28:12.565066 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:12.565108 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:12.565162 (+ 54us) server_negotiation.cc:407] Connection header received
0618 22:28:12.566774 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:12.566797 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:12.567228 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:12.567599 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:12.568643 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:12.568649 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:12.568661 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:28:12.568674 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:12.568744 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:28:12.568760 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:12.568919 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:12.569020 (+ 101us) server_negotiation.cc:299] Negotiation successful
0618 22:28:12.569305 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":394,"threads_started":1}
I20260618 22:28:12.570539 21043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f: Neither blocks nor log segments found. Creating new log.
I20260618 22:28:12.571636 21043 log.cc:824] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f: Log is configured to *not* fsync() on all Append() calls
I20260618 22:28:12.574447 21043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f: No bootstrap required, opened a new log
I20260618 22:28:12.581686 21043 raft_consensus.cc:348] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a324a4f89bc1495e9f8a0afadb38a25f" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45967 } }
I20260618 22:28:12.582145 21043 raft_consensus.cc:374] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:28:12.582249 21043 raft_consensus.cc:729] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a324a4f89bc1495e9f8a0afadb38a25f, State: Initialized, Role: FOLLOWER
I20260618 22:28:12.582857 21043 consensus_queue.cc:260] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a324a4f89bc1495e9f8a0afadb38a25f" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45967 } }
I20260618 22:28:12.583062 21043 raft_consensus.cc:388] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:28:12.583154 21043 raft_consensus.cc:482] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:28:12.583267 21043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:28:12.585409 21043 raft_consensus.cc:504] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a324a4f89bc1495e9f8a0afadb38a25f" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45967 } }
I20260618 22:28:12.585856 21043 leader_election.cc:302] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a324a4f89bc1495e9f8a0afadb38a25f; no voters:
I20260618 22:28:12.586565 21043 leader_election.cc:288] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:28:12.586675 21048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:28:12.587440 21048 raft_consensus.cc:686] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f [term 1 LEADER]: Becoming Leader. State: Replica: a324a4f89bc1495e9f8a0afadb38a25f, State: Running, Role: LEADER
I20260618 22:28:12.587966 21048 consensus_queue.cc:237] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a324a4f89bc1495e9f8a0afadb38a25f" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45967 } }
I20260618 22:28:12.588647 21043 sys_catalog.cc:564] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:28:12.590380 21050 sys_catalog.cc:455] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a324a4f89bc1495e9f8a0afadb38a25f. Latest consensus state: current_term: 1 leader_uuid: "a324a4f89bc1495e9f8a0afadb38a25f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a324a4f89bc1495e9f8a0afadb38a25f" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45967 } } }
I20260618 22:28:12.590463 21049 sys_catalog.cc:455] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a324a4f89bc1495e9f8a0afadb38a25f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a324a4f89bc1495e9f8a0afadb38a25f" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45967 } } }
I20260618 22:28:12.590737 21049 sys_catalog.cc:458] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f [sys.catalog]: This master's current role is: LEADER
I20260618 22:28:12.590741 21050 sys_catalog.cc:458] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f [sys.catalog]: This master's current role is: LEADER
I20260618 22:28:12.592592 21055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:28:12.597512 21055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:28:12.606801 21055 catalog_manager.cc:1347] Generated new cluster ID: 6056ff6a6d44469d89e30e909bbf0137
I20260618 22:28:12.606920 21055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:28:12.617921 21055 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:28:12.618412 21055 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:28:12.627835 21055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a324a4f89bc1495e9f8a0afadb38a25f: Generated new TSK 0
I20260618 22:28:12.628624 21055 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:28:12.644398 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45967
--builtin_ntp_servers=127.10.242.212:34331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260618 22:28:12.969727 21067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:28:12.970033 21067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:28:12.970141 21067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:28:12.977489 21067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:28:12.977591 21067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:28:12.977698 21067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:28:12.986696 21067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:28:12.987874 21067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:28:12.989703 21067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:28:13.002234 21075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:13.002310 21073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:13.002486 21072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:28:13.003209 21067 server_base.cc:1034] running on GCE node
I20260618 22:28:13.003849 21067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:28:13.004807 21067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:28:13.006027 21067 hybrid_clock.cc:648] HybridClock initialized: now 1781821693005976 us; error 43 us; skew 500 ppm
I20260618 22:28:13.006345 21067 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:28:13.008526 21067 webserver.cc:460] Webserver started at http://127.10.242.193:46865/ using document root <none> and password file <none>
I20260618 22:28:13.009259 21067 fs_manager.cc:359] Metadata directory not provided
I20260618 22:28:13.009366 21067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:28:13.009675 21067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:28:13.012519 21067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "65d2cf6c5f944487b0db9ff51365ee40"
format_stamp: "Formatted at 2026-06-18 22:28:13 on dist-test-slave-0956"
server_key: "60b8c7ee6285178152aec31c37b7b3f9"
server_key_iv: "deb38ebc01768cef72772a5023c74024"
server_key_version: "encryptionkey@0"
I20260618 22:28:13.013358 21067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "65d2cf6c5f944487b0db9ff51365ee40"
format_stamp: "Formatted at 2026-06-18 22:28:13 on dist-test-slave-0956"
server_key: "60b8c7ee6285178152aec31c37b7b3f9"
server_key_iv: "deb38ebc01768cef72772a5023c74024"
server_key_version: "encryptionkey@0"
I20260618 22:28:13.018291 21067 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:28:13.021699 21081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:13.023092 21067 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:28:13.023286 21067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "65d2cf6c5f944487b0db9ff51365ee40"
format_stamp: "Formatted at 2026-06-18 22:28:13 on dist-test-slave-0956"
server_key: "60b8c7ee6285178152aec31c37b7b3f9"
server_key_iv: "deb38ebc01768cef72772a5023c74024"
server_key_version: "encryptionkey@0"
I20260618 22:28:13.023494 21067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:28:13.028949 21067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:28:13.029790 21067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:28:13.030047 21067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:28:13.030963 21067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:28:13.032413 21067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:28:13.032505 21067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:13.032610 21067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:28:13.032682 21067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:13.061299 21067 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:39805
I20260618 22:28:13.061353 21193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:39805 every 8 connection(s)
I20260618 22:28:13.062561 21067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:28:13.064755 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 21067
I20260618 22:28:13.064934 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:28:13.065608 11211 external_mini_cluster.cc:1428] Setting key 4a92edc448af3dab7884e9361d9d99d3
I20260618 22:28:13.068029 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45967
--builtin_ntp_servers=127.10.242.212:34331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260618 22:28:13.070618 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:13.065124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:36813 (local address 127.10.242.254:45967)
0618 22:28:13.065315 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:13.065321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:13.066904 (+ 1583us) server_negotiation.cc:407] Connection header received
0618 22:28:13.068514 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:13.068520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:13.068581 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:13.068663 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:13.069956 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:13.069962 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:13.069967 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:28:13.069986 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:13.070015 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:28:13.070026 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:13.070129 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:13.070307 (+ 178us) server_negotiation.cc:299] Negotiation successful
0618 22:28:13.070370 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:28:13.071611 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:13.065677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45967 (local address 127.10.242.193:36813)
0618 22:28:13.066733 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:13.066784 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:13.068264 (+ 1480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:13.068905 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:13.068913 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:13.069008 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:13.069652 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:28:13.069655 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:13.069699 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:28:13.069784 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:28:13.070108 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:28:13.070118 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:28:13.070125 (+ 7us) client_negotiation.cc:769] Sending connection context
0618 22:28:13.070374 (+ 249us) client_negotiation.cc:240] Negotiation successful
0618 22:28:13.070695 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":292,"threads_started":1}
I20260618 22:28:13.076504 21194 heartbeater.cc:347] Connected to a master server at 127.10.242.254:45967
I20260618 22:28:13.076810 21194 heartbeater.cc:464] Registering TS with master...
I20260618 22:28:13.077451 21194 heartbeater.cc:511] Master 127.10.242.254:45967 requested a full tablet report, sending...
I20260618 22:28:13.079381 21008 ts_manager.cc:194] Registered new tserver with Master: 65d2cf6c5f944487b0db9ff51365ee40 (127.10.242.193:39805)
W20260618 22:28:13.381628 21197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:28:13.381919 21197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:28:13.382036 21197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:28:13.389093 21197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:28:13.389211 21197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:28:13.389339 21197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:28:13.398437 21197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:28:13.399659 21197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:28:13.401350 21197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:28:13.413971 21202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:13.413887 21205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:13.413921 21203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:28:13.414805 21197 server_base.cc:1034] running on GCE node
I20260618 22:28:13.415414 21197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:28:13.416353 21197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:28:13.417553 21197 hybrid_clock.cc:648] HybridClock initialized: now 1781821693417491 us; error 54 us; skew 500 ppm
I20260618 22:28:13.417860 21197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:28:13.420085 21197 webserver.cc:460] Webserver started at http://127.10.242.194:37663/ using document root <none> and password file <none>
I20260618 22:28:13.420765 21197 fs_manager.cc:359] Metadata directory not provided
I20260618 22:28:13.420869 21197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:28:13.421206 21197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:28:13.424001 21197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "7f8a610816dd419fbeca1b7a04bc0f19"
format_stamp: "Formatted at 2026-06-18 22:28:13 on dist-test-slave-0956"
server_key: "d30a56d77271564270f5d018a6a94f8c"
server_key_iv: "fff4ba15a89f23a921b2aaa49851568c"
server_key_version: "encryptionkey@0"
I20260618 22:28:13.424743 21197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "7f8a610816dd419fbeca1b7a04bc0f19"
format_stamp: "Formatted at 2026-06-18 22:28:13 on dist-test-slave-0956"
server_key: "d30a56d77271564270f5d018a6a94f8c"
server_key_iv: "fff4ba15a89f23a921b2aaa49851568c"
server_key_version: "encryptionkey@0"
I20260618 22:28:13.430028 21197 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260618 22:28:13.433609 21211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:13.434979 21197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260618 22:28:13.435127 21197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "7f8a610816dd419fbeca1b7a04bc0f19"
format_stamp: "Formatted at 2026-06-18 22:28:13 on dist-test-slave-0956"
server_key: "d30a56d77271564270f5d018a6a94f8c"
server_key_iv: "fff4ba15a89f23a921b2aaa49851568c"
server_key_version: "encryptionkey@0"
I20260618 22:28:13.435329 21197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:28:13.441098 21197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:28:13.441893 21197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:28:13.442135 21197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:28:13.443037 21197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:28:13.444383 21197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:28:13.444479 21197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:13.444588 21197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:28:13.444658 21197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:13.473500 21197 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:46783
I20260618 22:28:13.473543 21323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:46783 every 8 connection(s)
I20260618 22:28:13.474766 21197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:28:13.479844 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 21197
I20260618 22:28:13.480007 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:28:13.480551 11211 external_mini_cluster.cc:1428] Setting key f9207cfd585b7c685adffa328c8365a6
I20260618 22:28:13.483196 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:13.477258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:59039 (local address 127.10.242.254:45967)
0618 22:28:13.477436 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:13.477440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:13.478939 (+ 1499us) server_negotiation.cc:407] Connection header received
0618 22:28:13.481067 (+ 2128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:13.481071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:13.481166 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:13.481275 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:13.482578 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:13.482582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:13.482586 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:28:13.482600 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:13.482631 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:28:13.482642 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:13.482740 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:13.482947 (+ 207us) server_negotiation.cc:299] Negotiation successful
0618 22:28:13.482994 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:28:13.483582 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45967
--builtin_ntp_servers=127.10.242.212:34331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260618 22:28:13.484505 21326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:13.477866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45967 (local address 127.10.242.194:59039)
0618 22:28:13.478765 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:13.478807 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:13.480723 (+ 1916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:13.481510 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:13.481523 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:13.481658 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:13.482299 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:28:13.482301 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:13.482341 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:28:13.482423 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:28:13.482733 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:28:13.482743 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:28:13.482750 (+ 7us) client_negotiation.cc:769] Sending connection context
0618 22:28:13.483029 (+ 279us) client_negotiation.cc:240] Negotiation successful
0618 22:28:13.483302 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":272,"threads_started":1}
I20260618 22:28:13.485852 21324 heartbeater.cc:347] Connected to a master server at 127.10.242.254:45967
I20260618 22:28:13.486140 21324 heartbeater.cc:464] Registering TS with master...
I20260618 22:28:13.486765 21324 heartbeater.cc:511] Master 127.10.242.254:45967 requested a full tablet report, sending...
I20260618 22:28:13.488138 21008 ts_manager.cc:194] Registered new tserver with Master: 7f8a610816dd419fbeca1b7a04bc0f19 (127.10.242.194:46783)
W20260618 22:28:13.805028 21327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:28:13.805361 21327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:28:13.805451 21327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:28:13.812866 21327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:28:13.812985 21327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:28:13.813167 21327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:28:13.822433 21327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:28:13.823628 21327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:28:13.825402 21327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:28:13.838232 21332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:13.838164 21335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:13.838203 21333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:28:13.839295 21327 server_base.cc:1034] running on GCE node
I20260618 22:28:13.839977 21327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:28:13.840965 21327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:28:13.842175 21327 hybrid_clock.cc:648] HybridClock initialized: now 1781821693842122 us; error 69 us; skew 500 ppm
I20260618 22:28:13.842495 21327 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:28:13.844830 21327 webserver.cc:460] Webserver started at http://127.10.242.195:46667/ using document root <none> and password file <none>
I20260618 22:28:13.845585 21327 fs_manager.cc:359] Metadata directory not provided
I20260618 22:28:13.845672 21327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:28:13.845957 21327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:28:13.848805 21327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "2b9ca1014a464df3a8d9c79dd0929f0c"
format_stamp: "Formatted at 2026-06-18 22:28:13 on dist-test-slave-0956"
server_key: "85bd53fbc8ceb7d53fedf849ec28467d"
server_key_iv: "fd68214f2933ee9a389d8cef554ac240"
server_key_version: "encryptionkey@0"
I20260618 22:28:13.849608 21327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "2b9ca1014a464df3a8d9c79dd0929f0c"
format_stamp: "Formatted at 2026-06-18 22:28:13 on dist-test-slave-0956"
server_key: "85bd53fbc8ceb7d53fedf849ec28467d"
server_key_iv: "fd68214f2933ee9a389d8cef554ac240"
server_key_version: "encryptionkey@0"
I20260618 22:28:13.854756 21327 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:28:13.858250 21341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:13.859694 21327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260618 22:28:13.859830 21327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "2b9ca1014a464df3a8d9c79dd0929f0c"
format_stamp: "Formatted at 2026-06-18 22:28:13 on dist-test-slave-0956"
server_key: "85bd53fbc8ceb7d53fedf849ec28467d"
server_key_iv: "fd68214f2933ee9a389d8cef554ac240"
server_key_version: "encryptionkey@0"
I20260618 22:28:13.860019 21327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:28:13.865446 21327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:28:13.866191 21327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:28:13.866410 21327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:28:13.867254 21327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:28:13.868575 21327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:28:13.868659 21327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:13.868738 21327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:28:13.868788 21327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:13.897758 21327 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:44897
I20260618 22:28:13.897804 21453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:44897 every 8 connection(s)
I20260618 22:28:13.899017 21327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:28:13.905787 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 21327
I20260618 22:28:13.905934 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:28:13.906381 11211 external_mini_cluster.cc:1428] Setting key af9779d1e2e49dff15c7d263c6026c57
I20260618 22:28:13.907308 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:13.901537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:60355 (local address 127.10.242.254:45967)
0618 22:28:13.901725 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:13.901730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:13.903160 (+ 1430us) server_negotiation.cc:407] Connection header received
0618 22:28:13.904919 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:13.904926 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:13.904995 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:13.905139 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:13.906529 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:13.906536 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:13.906542 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:28:13.906564 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:13.906600 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:28:13.906615 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:13.906733 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:13.906963 (+ 230us) server_negotiation.cc:299] Negotiation successful
0618 22:28:13.907020 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:28:13.908540 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:13.902011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45967 (local address 127.10.242.195:60355)
0618 22:28:13.902977 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:13.903025 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:13.904631 (+ 1606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:13.905382 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:13.905391 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:13.905488 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:13.906199 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:28:13.906202 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:13.906242 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:28:13.906338 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:28:13.906710 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:28:13.906721 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:28:13.906729 (+ 8us) client_negotiation.cc:769] Sending connection context
0618 22:28:13.907027 (+ 298us) client_negotiation.cc:240] Negotiation successful
0618 22:28:13.907278 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":280,"threads_started":1}
I20260618 22:28:13.909981 21454 heartbeater.cc:347] Connected to a master server at 127.10.242.254:45967
I20260618 22:28:13.910233 21454 heartbeater.cc:464] Registering TS with master...
I20260618 22:28:13.910753 21454 heartbeater.cc:511] Master 127.10.242.254:45967 requested a full tablet report, sending...
I20260618 22:28:13.912012 21007 ts_manager.cc:194] Registered new tserver with Master: 2b9ca1014a464df3a8d9c79dd0929f0c (127.10.242.195:44897)
I20260618 22:28:13.921936 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260618 22:28:13.927575 21046 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:28:13.926163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45502 (local address 127.10.242.254:45967)
0618 22:28:13.926325 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:13.926330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:13.926800 (+ 470us) server_negotiation.cc:407] Connection header received
0618 22:28:13.926950 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:13.926954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:13.927004 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:13.927083 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:13.927358 (+ 275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45502: BlockingRecv error: recv got EOF from 127.0.0.1:45502 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:28:13.928509 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 21067
I20260618 22:28:13.931811 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:13.929404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46416 (local address 127.10.242.193:39805)
0618 22:28:13.929917 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:13.929926 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:13.929954 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:28:13.930008 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:13.930014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:13.930223 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:13.930350 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:13.931186 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:13.931193 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:13.931204 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:28:13.931220 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:13.931260 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:28:13.931274 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:13.931379 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:13.931465 (+ 86us) server_negotiation.cc:299] Negotiation successful
0618 22:28:13.931562 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":250,"threads_started":1}
I20260618 22:28:13.933002 21189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:28:14.013132 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 21067
I20260618 22:28:14.025916 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 21197
I20260618 22:28:14.029245 21465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:14.026797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40718 (local address 127.10.242.194:46783)
0618 22:28:14.027300 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:14.027332 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:14.027359 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:28:14.027427 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:14.027433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:14.027647 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:14.027770 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:14.028611 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:14.028618 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:14.028629 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:28:14.028641 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:14.028687 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:28:14.028702 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:14.028802 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:14.028854 (+ 52us) server_negotiation.cc:299] Negotiation successful
0618 22:28:14.028938 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":244,"threads_started":1}
I20260618 22:28:14.030619 21319 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:28:14.111603 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 21197
I20260618 22:28:14.123617 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 21327
I20260618 22:28:14.127753 21467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:14.124596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44316 (local address 127.10.242.195:44897)
0618 22:28:14.125294 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:14.125302 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:14.125334 (+ 32us) server_negotiation.cc:407] Connection header received
0618 22:28:14.125399 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:14.125406 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:14.125627 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:14.125789 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:14.126954 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:14.126961 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:14.126973 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:28:14.126990 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:14.127034 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:28:14.127051 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:14.127219 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:14.127330 (+ 111us) server_negotiation.cc:299] Negotiation successful
0618 22:28:14.127430 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":330,"threads_started":1}
I20260618 22:28:14.128999 21449 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:28:14.211179 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 21327
I20260618 22:28:14.224880 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 20977
I20260618 22:28:14.226068 21038 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:28:14.324723 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 20977
2026-06-18T22:28:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2227 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:28:14 dist-test-slave-0956 krb5kdc[21472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:28:14 dist-test-slave-0956 krb5kdc[21472](info): set up 2 sockets
Jun 18 22:28:14 dist-test-slave-0956 krb5kdc[21472](info): commencing operation
krb5kdc: starting...
W20260618 22:28:15.333806 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.973s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:28:15 dist-test-slave-0956 krb5kdc[21472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:28:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:28:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:28:15.549877 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:39471
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:39471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:28:15.869161 21485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:28:15.869462 21485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:28:15.869551 21485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:28:15.876801 21485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:28:15.876926 21485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:28:15.876973 21485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:28:15.877028 21485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:28:15.877069 21485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:28:15.887154 21485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:39471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:39471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:28:15.888422 21485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:28:15.889724 21485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:28:15.901283 21490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:15.901165 21491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:15.901186 21493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:28:15.901973 21485 server_base.cc:1034] running on GCE node
I20260618 22:28:15.902732 21485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:28:15.904217 21485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:28:15.905419 21485 hybrid_clock.cc:648] HybridClock initialized: now 1781821695905358 us; error 54 us; skew 500 ppm
I20260618 22:28:15.905762 21485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:28:15 dist-test-slave-0956 krb5kdc[21472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821695, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:28:15.908977 21485 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:28:15.910641 21485 webserver.cc:460] Webserver started at http://127.10.242.254:40069/ using document root <none> and password file <none>
I20260618 22:28:15.911361 21485 fs_manager.cc:359] Metadata directory not provided
I20260618 22:28:15.911489 21485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:28:15.911788 21485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:28:15.914858 21485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "c6f9e86cbc3544b0b8834023d9623b39"
format_stamp: "Formatted at 2026-06-18 22:28:15 on dist-test-slave-0956"
server_key: "537c0c554d3810993b515b194d6d31ad"
server_key_iv: "fa37f7cd48af2657334693758d7a9131"
server_key_version: "encryptionkey@0"
I20260618 22:28:15.915642 21485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "c6f9e86cbc3544b0b8834023d9623b39"
format_stamp: "Formatted at 2026-06-18 22:28:15 on dist-test-slave-0956"
server_key: "537c0c554d3810993b515b194d6d31ad"
server_key_iv: "fa37f7cd48af2657334693758d7a9131"
server_key_version: "encryptionkey@0"
I20260618 22:28:15.920964 21485 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:28:15.924444 21500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:15.925859 21485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:28:15.926033 21485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "c6f9e86cbc3544b0b8834023d9623b39"
format_stamp: "Formatted at 2026-06-18 22:28:15 on dist-test-slave-0956"
server_key: "537c0c554d3810993b515b194d6d31ad"
server_key_iv: "fa37f7cd48af2657334693758d7a9131"
server_key_version: "encryptionkey@0"
I20260618 22:28:15.926215 21485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:28:15.950425 21485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:28:15.954519 21485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:28:15.954797 21485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:28:15.972388 21485 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:39471
I20260618 22:28:15.972425 21551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:39471 every 8 connection(s)
I20260618 22:28:15.973805 21485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:28:15.977625 21552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:28:15.981258 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 21485
I20260618 22:28:15.981467 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:28:15.982245 11211 external_mini_cluster.cc:1428] Setting key 7956267f67123ab3117b713367471b87
I20260618 22:28:15.990722 21552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39: Bootstrap starting.
I20260618 22:28:15.994642 21552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39: Neither blocks nor log segments found. Creating new log.
I20260618 22:28:15.995985 21552 log.cc:824] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39: Log is configured to *not* fsync() on all Append() calls
I20260618 22:28:16.001025 21552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39: No bootstrap required, opened a new log
Jun 18 22:28:16 dist-test-slave-0956 krb5kdc[21472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:28:16.008880 21552 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6f9e86cbc3544b0b8834023d9623b39" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39471 } }
I20260618 22:28:16.009344 21552 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:28:16.009452 21552 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6f9e86cbc3544b0b8834023d9623b39, State: Initialized, Role: FOLLOWER
I20260618 22:28:16.010037 21552 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6f9e86cbc3544b0b8834023d9623b39" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39471 } }
I20260618 22:28:16.010243 21552 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:28:16.010366 21552 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:28:16.010514 21552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:28:16.010581 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:15.984295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41880 (local address 127.10.242.254:39471)
0618 22:28:15.985051 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:15.985062 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:15.985141 (+ 79us) server_negotiation.cc:407] Connection header received
0618 22:28:15.986642 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:15.986665 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:15.987105 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:15.987450 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:15.989179 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:15.990637 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:15.992507 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:15.993530 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:15.993698 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:16.002468 (+ 8770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:16.002478 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:16.002502 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:28:16.002553 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:16.006018 (+ 3465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:28:16.006921 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:28:16.006928 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:28:16.006936 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:28:16.007121 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:28:16.007493 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:28:16.007497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:28:16.007499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:28:16.008085 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:16.008391 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:16.008820 (+ 429us) server_negotiation.cc:299] Negotiation successful
0618 22:28:16.009293 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":343,"threads_started":1}
I20260618 22:28:16.013311 21552 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6f9e86cbc3544b0b8834023d9623b39" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39471 } }
I20260618 22:28:16.013844 21552 leader_election.cc:302] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6f9e86cbc3544b0b8834023d9623b39; no voters:
I20260618 22:28:16.014379 21552 leader_election.cc:288] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:28:16.014504 21557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:28:16.015123 21557 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39 [term 1 LEADER]: Becoming Leader. State: Replica: c6f9e86cbc3544b0b8834023d9623b39, State: Running, Role: LEADER
I20260618 22:28:16.015690 21557 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6f9e86cbc3544b0b8834023d9623b39" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39471 } }
I20260618 22:28:16.016281 21552 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:28:16.019390 21558 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6f9e86cbc3544b0b8834023d9623b39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6f9e86cbc3544b0b8834023d9623b39" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39471 } } }
I20260618 22:28:16.019767 21558 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39 [sys.catalog]: This master's current role is: LEADER
I20260618 22:28:16.021273 21559 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6f9e86cbc3544b0b8834023d9623b39. Latest consensus state: current_term: 1 leader_uuid: "c6f9e86cbc3544b0b8834023d9623b39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6f9e86cbc3544b0b8834023d9623b39" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39471 } } }
I20260618 22:28:16.021564 21559 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39 [sys.catalog]: This master's current role is: LEADER
I20260618 22:28:16.022074 21569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:28:16.026530 21569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:28:16.035012 21569 catalog_manager.cc:1347] Generated new cluster ID: dd763e501bc645e8a38887a1857e6e7c
I20260618 22:28:16.035146 21569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:28:16.058976 21569 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:28:16.060314 21569 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:28:16.066861 21569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39: Generated new TSK 0
I20260618 22:28:16.067813 21569 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
I20260618 22:28:16.132802 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:39471
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:28:16.458576 21578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:28:16.458849 21578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:28:16.458963 21578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:28:16.466202 21578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:28:16.466311 21578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:28:16.466456 21578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:28:16.476107 21578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:39471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:28:16.477447 21578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:28:16.478708 21578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:28:16.490973 21584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:16.491029 21586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:16.491048 21583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:28:16.491680 21578 server_base.cc:1034] running on GCE node
I20260618 22:28:16.492250 21578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:28:16.493176 21578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:28:16.494380 21578 hybrid_clock.cc:648] HybridClock initialized: now 1781821696494319 us; error 55 us; skew 500 ppm
I20260618 22:28:16.494714 21578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:28:16 dist-test-slave-0956 krb5kdc[21472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821696, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:28:16.497701 21578 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:28:16.498998 21578 webserver.cc:460] Webserver started at http://127.10.242.193:33259/ using document root <none> and password file <none>
I20260618 22:28:16.499702 21578 fs_manager.cc:359] Metadata directory not provided
I20260618 22:28:16.499802 21578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:28:16.500042 21578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:28:16.502910 21578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "a38bd2d2e0db4f548537823642ec40f4"
format_stamp: "Formatted at 2026-06-18 22:28:16 on dist-test-slave-0956"
server_key: "612c11cd8c2632a109d3af7dd770d593"
server_key_iv: "1f85ec1859e5a9d60f1c0c11c0a0de4e"
server_key_version: "encryptionkey@0"
I20260618 22:28:16.503643 21578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "a38bd2d2e0db4f548537823642ec40f4"
format_stamp: "Formatted at 2026-06-18 22:28:16 on dist-test-slave-0956"
server_key: "612c11cd8c2632a109d3af7dd770d593"
server_key_iv: "1f85ec1859e5a9d60f1c0c11c0a0de4e"
server_key_version: "encryptionkey@0"
I20260618 22:28:16.508718 21578 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:28:16.512233 21593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:16.513654 21578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:28:16.513798 21578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "a38bd2d2e0db4f548537823642ec40f4"
format_stamp: "Formatted at 2026-06-18 22:28:16 on dist-test-slave-0956"
server_key: "612c11cd8c2632a109d3af7dd770d593"
server_key_iv: "1f85ec1859e5a9d60f1c0c11c0a0de4e"
server_key_version: "encryptionkey@0"
I20260618 22:28:16.513980 21578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:28:16.534057 21578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:28:16.538892 21578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:28:16.539260 21578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:28:16.540306 21578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:28:16.541762 21578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:28:16.541844 21578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:16.541946 21578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:28:16.541994 21578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:16.571293 21578 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:46225
I20260618 22:28:16.571295 21705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:46225 every 8 connection(s)
I20260618 22:28:16.572608 21578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:28:16.577853 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 21578
I20260618 22:28:16.578047 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:28:16.578784 11211 external_mini_cluster.cc:1428] Setting key 4b063be7a60c188b23f98557fd5affb9
Jun 18 22:28:16 dist-test-slave-0956 krb5kdc[21472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821696, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:28:16.595979 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:16.575125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:53293 (local address 127.10.242.254:39471)
0618 22:28:16.575524 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:16.575528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:16.576505 (+ 977us) server_negotiation.cc:407] Connection header received
0618 22:28:16.578649 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:16.578653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:16.578719 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:16.578809 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:16.580562 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:16.581582 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:16.583080 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:16.583583 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:16.583696 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:16.590296 (+ 6600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:16.590303 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:16.590308 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:28:16.590346 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:16.593141 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:28:16.593805 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:28:16.593809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:28:16.593813 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:28:16.593944 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:28:16.594294 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:28:16.594297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:28:16.594300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:28:16.594605 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:16.594733 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:16.595381 (+ 648us) server_negotiation.cc:299] Negotiation successful
0618 22:28:16.595614 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":196,"threads_started":1}
I20260618 22:28:16.596869 21709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:16.575675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:39471 (local address 127.10.242.193:53293)
0618 22:28:16.576347 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:16.576391 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:16.578358 (+ 1967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:16.579089 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:16.579100 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:16.579670 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:16.580402 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:16.580427 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:16.581716 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:16.581720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:16.582944 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:16.582956 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:16.583763 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:16.583767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:16.583957 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:16.587542 (+ 3585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:16.587579 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:16.590050 (+ 2471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:28:16.593275 (+ 3225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:28:16.593283 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:28:16.593296 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:28:16.593701 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:28:16.594034 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:28:16.594037 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:28:16.594039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:28:16.594203 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:28:16.594701 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:28:16.594708 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:28:16.595037 (+ 329us) client_negotiation.cc:769] Sending connection context
0618 22:28:16.595271 (+ 234us) client_negotiation.cc:240] Negotiation successful
0618 22:28:16.595721 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":251,"threads_started":1}
I20260618 22:28:16.598430 21706 heartbeater.cc:347] Connected to a master server at 127.10.242.254:39471
I20260618 22:28:16.598701 21706 heartbeater.cc:464] Registering TS with master...
I20260618 22:28:16.599390 21706 heartbeater.cc:511] Master 127.10.242.254:39471 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.242.194@KRBTEST.COM; defaulting to no policy
I20260618 22:28:16.601424 21517 ts_manager.cc:194] Registered new tserver with Master: a38bd2d2e0db4f548537823642ec40f4 (127.10.242.193:46225)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.194@KRBTEST.COM" created.
I20260618 22:28:16.603055 21517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:53293
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
I20260618 22:28:16.627592 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:39471
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:28:16.949373 21712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:28:16.949645 21712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:28:16.949784 21712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:28:16.957005 21712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:28:16.957170 21712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:28:16.957336 21712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:28:16.967136 21712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:39471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:28:16.968392 21712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:28:16.969698 21712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:28:16.982333 21717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:16.982281 21718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:16.982959 21720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:28:16.983788 21712 server_base.cc:1034] running on GCE node
I20260618 22:28:16.984403 21712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:28:16.985399 21712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:28:16.986618 21712 hybrid_clock.cc:648] HybridClock initialized: now 1781821696986554 us; error 56 us; skew 500 ppm
I20260618 22:28:16.986938 21712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:28:16 dist-test-slave-0956 krb5kdc[21472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821696, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:28:16.990289 21712 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:28:16.991894 21712 webserver.cc:460] Webserver started at http://127.10.242.194:36019/ using document root <none> and password file <none>
I20260618 22:28:16.992822 21712 fs_manager.cc:359] Metadata directory not provided
I20260618 22:28:16.992933 21712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:28:16.993310 21712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:28:16.996253 21712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "5a3971e723334ce6b52e0ee48e68a00b"
format_stamp: "Formatted at 2026-06-18 22:28:16 on dist-test-slave-0956"
server_key: "9dfe10aab971b17e1d34df5425fe2bf1"
server_key_iv: "52b7ab40a85cfff1e955e26d3d9ad9ce"
server_key_version: "encryptionkey@0"
I20260618 22:28:16.997015 21712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "5a3971e723334ce6b52e0ee48e68a00b"
format_stamp: "Formatted at 2026-06-18 22:28:16 on dist-test-slave-0956"
server_key: "9dfe10aab971b17e1d34df5425fe2bf1"
server_key_iv: "52b7ab40a85cfff1e955e26d3d9ad9ce"
server_key_version: "encryptionkey@0"
I20260618 22:28:17.002166 21712 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260618 22:28:17.005488 21727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:17.006851 21712 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:28:17.006987 21712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "5a3971e723334ce6b52e0ee48e68a00b"
format_stamp: "Formatted at 2026-06-18 22:28:16 on dist-test-slave-0956"
server_key: "9dfe10aab971b17e1d34df5425fe2bf1"
server_key_iv: "52b7ab40a85cfff1e955e26d3d9ad9ce"
server_key_version: "encryptionkey@0"
I20260618 22:28:17.007165 21712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:28:17.018790 21712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:28:17.022794 21712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:28:17.023061 21712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:28:17.023969 21712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:28:17.025359 21712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:28:17.025439 21712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:17.025556 21712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:28:17.025602 21712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:17.055089 21712 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:41369
I20260618 22:28:17.055111 21839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:41369 every 8 connection(s)
I20260618 22:28:17.056315 21712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:28:17.058135 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 21712
I20260618 22:28:17.058305 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:28:17.058755 11211 external_mini_cluster.cc:1428] Setting key b7d43a80935b9b54371ef57e0fd401db
Jun 18 22:28:17 dist-test-slave-0956 krb5kdc[21472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821696, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:28:17.075248 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:17.058930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:43999 (local address 127.10.242.254:39471)
0618 22:28:17.059122 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:17.059128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:17.060401 (+ 1273us) server_negotiation.cc:407] Connection header received
0618 22:28:17.062225 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:17.062231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:17.062309 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:17.062395 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:17.063837 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.064542 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.066153 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.066551 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.066661 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:17.070030 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:17.070035 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:17.070038 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:28:17.070070 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:17.072383 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:28:17.073072 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:28:17.073075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:28:17.073099 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:28:17.073226 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:28:17.073627 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:28:17.073631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:28:17.073634 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:28:17.073898 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:17.074028 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:17.074707 (+ 679us) server_negotiation.cc:299] Negotiation successful
0618 22:28:17.074925 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260618 22:28:17.076229 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:17.059497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:39471 (local address 127.10.242.194:43999)
0618 22:28:17.060248 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:17.060299 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:17.061968 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:17.062631 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:17.062640 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:17.063116 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:17.063681 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:17.063698 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.064683 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.064687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:17.066015 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:17.066031 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.066757 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.066764 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:17.066996 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:17.067580 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:17.067602 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:17.069817 (+ 2215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:28:17.072519 (+ 2702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:28:17.072528 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:28:17.072545 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:28:17.072956 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:28:17.073328 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:28:17.073333 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:28:17.073337 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:28:17.073516 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:28:17.074019 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:28:17.074027 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:28:17.074353 (+ 326us) client_negotiation.cc:769] Sending connection context
0618 22:28:17.074588 (+ 235us) client_negotiation.cc:240] Negotiation successful
0618 22:28:17.075120 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":313,"threads_started":1}
WARNING: no policy specified for kudu/127.10.242.195@KRBTEST.COM; defaulting to no policy
I20260618 22:28:17.077663 21840 heartbeater.cc:347] Connected to a master server at 127.10.242.254:39471
I20260618 22:28:17.077951 21840 heartbeater.cc:464] Registering TS with master...
I20260618 22:28:17.078713 21840 heartbeater.cc:511] Master 127.10.242.254:39471 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.195@KRBTEST.COM" created.
I20260618 22:28:17.080268 21517 ts_manager.cc:194] Registered new tserver with Master: 5a3971e723334ce6b52e0ee48e68a00b (127.10.242.194:41369)
I20260618 22:28:17.081125 21517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:43999
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
I20260618 22:28:17.104540 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:39471
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:28:17.418321 21845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:28:17.418643 21845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:28:17.418797 21845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:28:17.426065 21845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:28:17.426172 21845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:28:17.426349 21845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:28:17.436496 21845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:39471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:28:17.437821 21845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:28:17.439123 21845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:28:17.452654 21851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:17.452664 21850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:28:17.453238 21853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:28:17.453305 21845 server_base.cc:1034] running on GCE node
I20260618 22:28:17.453915 21845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:28:17.454857 21845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:28:17.456053 21845 hybrid_clock.cc:648] HybridClock initialized: now 1781821697456009 us; error 36 us; skew 500 ppm
I20260618 22:28:17.456321 21845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:28:17 dist-test-slave-0956 krb5kdc[21472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821697, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:28:17.459251 21845 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:28:17.460707 21845 webserver.cc:460] Webserver started at http://127.10.242.195:35965/ using document root <none> and password file <none>
I20260618 22:28:17.461406 21845 fs_manager.cc:359] Metadata directory not provided
I20260618 22:28:17.461529 21845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:28:17.461798 21845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:28:17.464622 21845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "36124b95fa794afd8a07b02242dd6ee8"
format_stamp: "Formatted at 2026-06-18 22:28:17 on dist-test-slave-0956"
server_key: "9d1d6b86e31975379e089f79b73687da"
server_key_iv: "4139e8611aad30a4f588562ad99dd575"
server_key_version: "encryptionkey@0"
I20260618 22:28:17.465360 21845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "36124b95fa794afd8a07b02242dd6ee8"
format_stamp: "Formatted at 2026-06-18 22:28:17 on dist-test-slave-0956"
server_key: "9d1d6b86e31975379e089f79b73687da"
server_key_iv: "4139e8611aad30a4f588562ad99dd575"
server_key_version: "encryptionkey@0"
I20260618 22:28:17.470269 21845 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:28:17.473557 21860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:17.474889 21845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:28:17.475032 21845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "36124b95fa794afd8a07b02242dd6ee8"
format_stamp: "Formatted at 2026-06-18 22:28:17 on dist-test-slave-0956"
server_key: "9d1d6b86e31975379e089f79b73687da"
server_key_iv: "4139e8611aad30a4f588562ad99dd575"
server_key_version: "encryptionkey@0"
I20260618 22:28:17.475198 21845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:28:17.509191 21845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:28:17.513520 21845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:28:17.513797 21845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:28:17.514730 21845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:28:17.516052 21845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:28:17.516124 21845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:17.516230 21845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:28:17.516288 21845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:28:17.545280 21845 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:39685
I20260618 22:28:17.545363 21972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:39685 every 8 connection(s)
I20260618 22:28:17.546515 21845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:28:17.555809 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 21845
I20260618 22:28:17.556000 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:28:17.556493 11211 external_mini_cluster.cc:1428] Setting key b73741acc9335f1db422b5539d1cadf0
Jun 18 22:28:17 dist-test-slave-0956 krb5kdc[21472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821697, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:28:17.565577 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:17.548906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:57007 (local address 127.10.242.254:39471)
0618 22:28:17.549101 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:17.549108 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:17.550256 (+ 1148us) server_negotiation.cc:407] Connection header received
0618 22:28:17.552084 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:17.552090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:17.552162 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:17.552248 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:17.553758 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.554496 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.556328 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.556776 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.556882 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:17.560579 (+ 3697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:17.560584 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:17.560587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:28:17.560620 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:17.563005 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:28:17.563686 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:28:17.563689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:28:17.563692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:28:17.563811 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:28:17.564131 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:28:17.564134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:28:17.564136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:28:17.564398 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:17.564512 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:17.565146 (+ 634us) server_negotiation.cc:299] Negotiation successful
0618 22:28:17.565308 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:28:17.566603 21975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:17.549447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:39471 (local address 127.10.242.195:57007)
0618 22:28:17.550105 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:17.550148 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:17.551784 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:17.552493 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:17.552502 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:17.552986 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:28:17.553595 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:17.553613 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.554631 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.554635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:17.556175 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:17.556191 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.557014 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.557020 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:17.557298 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:17.558028 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:28:17.558064 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:28:17.560386 (+ 2322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:28:17.563133 (+ 2747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:28:17.563141 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:28:17.563155 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:28:17.563591 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:28:17.563884 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:28:17.563888 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:28:17.563890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:28:17.564053 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:28:17.564499 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:28:17.564504 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:28:17.564819 (+ 315us) client_negotiation.cc:769] Sending connection context
0618 22:28:17.565033 (+ 214us) client_negotiation.cc:240] Negotiation successful
0618 22:28:17.565468 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":249,"threads_started":1}
I20260618 22:28:17.567916 21973 heartbeater.cc:347] Connected to a master server at 127.10.242.254:39471
I20260618 22:28:17.568167 21973 heartbeater.cc:464] Registering TS with master...
I20260618 22:28:17.568840 21973 heartbeater.cc:511] Master 127.10.242.254:39471 requested a full tablet report, sending...
I20260618 22:28:17.570206 21517 ts_manager.cc:194] Registered new tserver with Master: 36124b95fa794afd8a07b02242dd6ee8 (127.10.242.195:39685)
I20260618 22:28:17.571174 21517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:57007
I20260618 22:28:17.571717 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:28:17.587127 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:17.576128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41888 (local address 127.10.242.254:39471)
0618 22:28:17.576319 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:17.576325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:17.576695 (+ 370us) server_negotiation.cc:407] Connection header received
0618 22:28:17.576818 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:17.576821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:17.576879 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:17.576963 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:17.578346 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.579068 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.580205 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.580593 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.580683 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:17.582702 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:17.582707 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:17.582710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:28:17.582739 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:17.585075 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:28:17.585630 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:28:17.585634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:28:17.585636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:28:17.585753 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:28:17.586014 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:28:17.586017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:28:17.586019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:28:17.586296 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:17.586480 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:17.586708 (+ 228us) server_negotiation.cc:299] Negotiation successful
0618 22:28:17.586855 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:28:17.590811 21517 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41888:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:28:17.597404 21517 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:28:17.606570 21706 heartbeater.cc:503] Master 127.10.242.254:39471 was elected leader, sending a full tablet report...
I20260618 22:28:17.631920 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:17.609184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:39685 (local address 127.0.0.1:37148)
0618 22:28:17.622507 (+ 13323us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:17.622523 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:17.622628 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:17.623566 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:17.623571 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:17.623598 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:28:17.624049 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:17.624060 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.626138 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.626144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:17.628522 (+ 2378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:17.628539 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.630821 (+ 2282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.630827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:17.631052 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:17.631100 (+ 48us) client_negotiation.cc:769] Sending connection context
0618 22:28:17.631236 (+ 136us) client_negotiation.cc:240] Negotiation successful
0618 22:28:17.631451 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13203,"mutex_wait_us":228,"thread_start_us":12823,"threads_started":1}
I20260618 22:28:17.632130 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:17.609565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60064 (local address 127.10.242.194:41369)
0618 22:28:17.621742 (+ 12177us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:17.621754 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:17.622067 (+ 313us) server_negotiation.cc:407] Connection header received
0618 22:28:17.622182 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:17.622192 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:17.622488 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:17.622680 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:28:17.623638 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.625401 (+ 1763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.628498 (+ 3097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.630254 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.630439 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:17.630623 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:17.631654 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0618 22:28:17.631838 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12013,"thread_start_us":11866,"threads_started":1}
I20260618 22:28:17.632639 21987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:17.609411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37148 (local address 127.10.242.195:39685)
0618 22:28:17.622138 (+ 12727us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:17.622146 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:17.622744 (+ 598us) server_negotiation.cc:407] Connection header received
0618 22:28:17.622847 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:17.622854 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:17.623101 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:17.623288 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:28:17.624273 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.625972 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.628723 (+ 2751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.630246 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.630439 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:17.630622 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:17.632236 (+ 1614us) server_negotiation.cc:299] Negotiation successful
0618 22:28:17.632393 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12562,"thread_start_us":12400,"threads_started":1}
I20260618 22:28:17.632819 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:17.609182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:41369 (local address 127.0.0.1:60064)
0618 22:28:17.621668 (+ 12486us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:17.621724 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:17.621965 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:17.622969 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:17.622976 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:17.623012 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:28:17.623430 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:17.623444 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.625588 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.625595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:17.628285 (+ 2690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:17.628304 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.631234 (+ 2930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.631241 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:17.631433 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:17.631468 (+ 35us) client_negotiation.cc:769] Sending connection context
0618 22:28:17.632315 (+ 847us) client_negotiation.cc:240] Negotiation successful
0618 22:28:17.632456 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12191,"thread_start_us":12020,"threads_started":1}
I20260618 22:28:17.636312 21775 tablet_service.cc:1467] Processing CreateTablet for tablet 8aa8a7953cd244cdb76ac856fb8a47cd (DEFAULT_TABLE table=test-table [id=8a8d04ab9b214c1aa03401f2e03ed8e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:28:17.636299 21908 tablet_service.cc:1467] Processing CreateTablet for tablet 8aa8a7953cd244cdb76ac856fb8a47cd (DEFAULT_TABLE table=test-table [id=8a8d04ab9b214c1aa03401f2e03ed8e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:28:17.637528 21908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aa8a7953cd244cdb76ac856fb8a47cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:28:17.637558 21775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aa8a7953cd244cdb76ac856fb8a47cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:28:17.643808 21990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:17.622965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53554 (local address 127.10.242.193:46225)
0618 22:28:17.631312 (+ 8347us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:17.631322 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:17.631898 (+ 576us) server_negotiation.cc:407] Connection header received
0618 22:28:17.632070 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:17.632079 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:17.632480 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:17.632700 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:28:17.633583 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.637976 (+ 4393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.640665 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.642163 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.642367 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:17.642547 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:17.643302 (+ 755us) server_negotiation.cc:299] Negotiation successful
0618 22:28:17.643480 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8145,"thread_start_us":7957,"threads_started":1}
I20260618 22:28:17.643994 21989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:17.622938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:46225 (local address 127.0.0.1:53554)
0618 22:28:17.631774 (+ 8836us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:17.631790 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:17.631927 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:17.632706 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:17.632711 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:17.632746 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:28:17.633129 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:17.633141 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.638508 (+ 5367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.638513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:17.640483 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:17.640500 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.642393 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.642399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:17.642581 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:17.642618 (+ 37us) client_negotiation.cc:769] Sending connection context
0618 22:28:17.642715 (+ 97us) client_negotiation.cc:240] Negotiation successful
0618 22:28:17.642890 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8690,"thread_start_us":8550,"threads_started":1}
I20260618 22:28:17.648138 21641 tablet_service.cc:1467] Processing CreateTablet for tablet 8aa8a7953cd244cdb76ac856fb8a47cd (DEFAULT_TABLE table=test-table [id=8a8d04ab9b214c1aa03401f2e03ed8e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:28:17.649740 21641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aa8a7953cd244cdb76ac856fb8a47cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:28:17.650805 21991 tablet_bootstrap.cc:492] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b: Bootstrap starting.
I20260618 22:28:17.653918 21991 tablet_bootstrap.cc:654] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b: Neither blocks nor log segments found. Creating new log.
I20260618 22:28:17.654755 21992 tablet_bootstrap.cc:492] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8: Bootstrap starting.
I20260618 22:28:17.655175 21991 log.cc:824] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:28:17.658059 21992 tablet_bootstrap.cc:654] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8: Neither blocks nor log segments found. Creating new log.
I20260618 22:28:17.658341 21991 tablet_bootstrap.cc:492] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b: No bootstrap required, opened a new log
I20260618 22:28:17.658619 21991 ts_tablet_manager.cc:1397] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260618 22:28:17.659085 21992 log.cc:824] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8: Log is configured to *not* fsync() on all Append() calls
I20260618 22:28:17.661613 21992 tablet_bootstrap.cc:492] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8: No bootstrap required, opened a new log
I20260618 22:28:17.661859 21992 ts_tablet_manager.cc:1397] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260618 22:28:17.664265 21995 tablet_bootstrap.cc:492] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4: Bootstrap starting.
I20260618 22:28:17.667726 21991 raft_consensus.cc:348] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36124b95fa794afd8a07b02242dd6ee8" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39685 } } peers { permanent_uuid: "5a3971e723334ce6b52e0ee48e68a00b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 41369 } } peers { permanent_uuid: "a38bd2d2e0db4f548537823642ec40f4" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46225 } }
I20260618 22:28:17.668078 21991 raft_consensus.cc:374] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:28:17.668183 21991 raft_consensus.cc:729] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3971e723334ce6b52e0ee48e68a00b, State: Initialized, Role: FOLLOWER
I20260618 22:28:17.668728 21991 consensus_queue.cc:260] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36124b95fa794afd8a07b02242dd6ee8" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39685 } } peers { permanent_uuid: "5a3971e723334ce6b52e0ee48e68a00b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 41369 } } peers { permanent_uuid: "a38bd2d2e0db4f548537823642ec40f4" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46225 } }
I20260618 22:28:17.668864 21995 tablet_bootstrap.cc:654] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4: Neither blocks nor log segments found. Creating new log.
I20260618 22:28:17.669703 21840 heartbeater.cc:503] Master 127.10.242.254:39471 was elected leader, sending a full tablet report...
I20260618 22:28:17.670117 21995 log.cc:824] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4: Log is configured to *not* fsync() on all Append() calls
I20260618 22:28:17.671646 21991 ts_tablet_manager.cc:1428] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260618 22:28:17.671363 21992 raft_consensus.cc:348] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36124b95fa794afd8a07b02242dd6ee8" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39685 } } peers { permanent_uuid: "5a3971e723334ce6b52e0ee48e68a00b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 41369 } } peers { permanent_uuid: "a38bd2d2e0db4f548537823642ec40f4" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46225 } }
I20260618 22:28:17.671775 21992 raft_consensus.cc:374] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:28:17.671883 21992 raft_consensus.cc:729] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36124b95fa794afd8a07b02242dd6ee8, State: Initialized, Role: FOLLOWER
I20260618 22:28:17.672487 21992 consensus_queue.cc:260] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36124b95fa794afd8a07b02242dd6ee8" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39685 } } peers { permanent_uuid: "5a3971e723334ce6b52e0ee48e68a00b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 41369 } } peers { permanent_uuid: "a38bd2d2e0db4f548537823642ec40f4" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46225 } }
I20260618 22:28:17.673493 21995 tablet_bootstrap.cc:492] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4: No bootstrap required, opened a new log
I20260618 22:28:17.673671 21973 heartbeater.cc:503] Master 127.10.242.254:39471 was elected leader, sending a full tablet report...
I20260618 22:28:17.673812 21995 ts_tablet_manager.cc:1397] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260618 22:28:17.674412 21992 ts_tablet_manager.cc:1428] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8: Time spent starting tablet: real 0.012s user 0.007s sys 0.006s
I20260618 22:28:17.681562 21995 raft_consensus.cc:348] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36124b95fa794afd8a07b02242dd6ee8" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39685 } } peers { permanent_uuid: "5a3971e723334ce6b52e0ee48e68a00b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 41369 } } peers { permanent_uuid: "a38bd2d2e0db4f548537823642ec40f4" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46225 } }
I20260618 22:28:17.681905 21995 raft_consensus.cc:374] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:28:17.681972 21995 raft_consensus.cc:729] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38bd2d2e0db4f548537823642ec40f4, State: Initialized, Role: FOLLOWER
I20260618 22:28:17.682462 21995 consensus_queue.cc:260] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36124b95fa794afd8a07b02242dd6ee8" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39685 } } peers { permanent_uuid: "5a3971e723334ce6b52e0ee48e68a00b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 41369 } } peers { permanent_uuid: "a38bd2d2e0db4f548537823642ec40f4" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46225 } }
I20260618 22:28:17.683795 21995 ts_tablet_manager.cc:1428] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260618 22:28:17.720775 21998 raft_consensus.cc:482] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:28:17.721097 21998 raft_consensus.cc:504] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36124b95fa794afd8a07b02242dd6ee8" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39685 } } peers { permanent_uuid: "5a3971e723334ce6b52e0ee48e68a00b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 41369 } } peers { permanent_uuid: "a38bd2d2e0db4f548537823642ec40f4" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46225 } }
I20260618 22:28:17.722751 21998 leader_election.cc:288] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a3971e723334ce6b52e0ee48e68a00b (127.10.242.194:41369), a38bd2d2e0db4f548537823642ec40f4 (127.10.242.193:46225)
I20260618 22:28:17.729018 21990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:17.723177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:40419 (local address 127.10.242.193:46225)
0618 22:28:17.723381 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:17.723387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:17.723404 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:28:17.723508 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:17.723514 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:17.723592 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:17.723688 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:28:17.724202 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.725186 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.727075 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.728240 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.728356 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:17.728411 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:17.728720 (+ 309us) server_negotiation.cc:299] Negotiation successful
0618 22:28:17.728809 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:28:17.729152 21975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:17.722993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:46225 (local address 127.10.242.195:40419)
0618 22:28:17.723165 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:17.723185 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:17.723301 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:17.723741 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:17.723750 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:17.723788 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:28:17.724037 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:17.724051 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.725308 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.725313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:17.726900 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:17.726925 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.728376 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.728382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:17.728569 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:17.728606 (+ 37us) client_negotiation.cc:769] Sending connection context
0618 22:28:17.728704 (+ 98us) client_negotiation.cc:240] Negotiation successful
0618 22:28:17.728843 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":3}
I20260618 22:28:17.730099 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:17.723195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:48365 (local address 127.10.242.194:41369)
0618 22:28:17.723357 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:17.723362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:17.724116 (+ 754us) server_negotiation.cc:407] Connection header received
0618 22:28:17.724275 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:17.724281 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:17.724352 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:17.724446 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:28:17.725021 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.726068 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.728080 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.729184 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.729330 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:17.729392 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:17.729705 (+ 313us) server_negotiation.cc:299] Negotiation successful
0618 22:28:17.729833 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260618 22:28:17.730159 22000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:17.723007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:41369 (local address 127.10.242.195:48365)
0618 22:28:17.724011 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0618 22:28:17.724028 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:28:17.724145 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:28:17.724475 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:28:17.724480 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:28:17.724504 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:28:17.724807 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:17.724817 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.726206 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.726209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:17.727917 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:28:17.727932 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.729389 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.729397 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:28:17.729563 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:17.729587 (+ 24us) client_negotiation.cc:769] Sending connection context
0618 22:28:17.729654 (+ 67us) client_negotiation.cc:240] Negotiation successful
0618 22:28:17.729739 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"mutex_wait_us":383,"thread_start_us":276,"threads_started":1}
I20260618 22:28:17.730429 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aa8a7953cd244cdb76ac856fb8a47cd" candidate_uuid: "36124b95fa794afd8a07b02242dd6ee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38bd2d2e0db4f548537823642ec40f4" is_pre_election: true
I20260618 22:28:17.730960 21661 raft_consensus.cc:2454] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36124b95fa794afd8a07b02242dd6ee8 in term 0.
I20260618 22:28:17.731031 21795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aa8a7953cd244cdb76ac856fb8a47cd" candidate_uuid: "36124b95fa794afd8a07b02242dd6ee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3971e723334ce6b52e0ee48e68a00b" is_pre_election: true
I20260618 22:28:17.731546 21795 raft_consensus.cc:2454] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36124b95fa794afd8a07b02242dd6ee8 in term 0.
I20260618 22:28:17.731618 21864 leader_election.cc:302] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36124b95fa794afd8a07b02242dd6ee8, a38bd2d2e0db4f548537823642ec40f4; no voters:
I20260618 22:28:17.731972 21998 raft_consensus.cc:2781] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:28:17.732125 21998 raft_consensus.cc:482] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:28:17.732231 21998 raft_consensus.cc:3037] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:28:17.734484 21998 raft_consensus.cc:504] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36124b95fa794afd8a07b02242dd6ee8" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39685 } } peers { permanent_uuid: "5a3971e723334ce6b52e0ee48e68a00b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 41369 } } peers { permanent_uuid: "a38bd2d2e0db4f548537823642ec40f4" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46225 } }
I20260618 22:28:17.735061 21998 leader_election.cc:288] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [CANDIDATE]: Term 1 election: Requested vote from peers 5a3971e723334ce6b52e0ee48e68a00b (127.10.242.194:41369), a38bd2d2e0db4f548537823642ec40f4 (127.10.242.193:46225)
I20260618 22:28:17.735445 21795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aa8a7953cd244cdb76ac856fb8a47cd" candidate_uuid: "36124b95fa794afd8a07b02242dd6ee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3971e723334ce6b52e0ee48e68a00b"
I20260618 22:28:17.735571 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aa8a7953cd244cdb76ac856fb8a47cd" candidate_uuid: "36124b95fa794afd8a07b02242dd6ee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38bd2d2e0db4f548537823642ec40f4"
I20260618 22:28:17.735625 21795 raft_consensus.cc:3037] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:28:17.735759 21661 raft_consensus.cc:3037] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:28:17.737872 21795 raft_consensus.cc:2454] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36124b95fa794afd8a07b02242dd6ee8 in term 1.
I20260618 22:28:17.737941 21661 raft_consensus.cc:2454] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36124b95fa794afd8a07b02242dd6ee8 in term 1.
I20260618 22:28:17.738281 21862 leader_election.cc:302] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36124b95fa794afd8a07b02242dd6ee8, 5a3971e723334ce6b52e0ee48e68a00b; no voters:
I20260618 22:28:17.738583 21998 raft_consensus.cc:2781] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:28:17.738997 21998 raft_consensus.cc:686] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [term 1 LEADER]: Becoming Leader. State: Replica: 36124b95fa794afd8a07b02242dd6ee8, State: Running, Role: LEADER
I20260618 22:28:17.739498 21998 consensus_queue.cc:237] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36124b95fa794afd8a07b02242dd6ee8" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39685 } } peers { permanent_uuid: "5a3971e723334ce6b52e0ee48e68a00b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 41369 } } peers { permanent_uuid: "a38bd2d2e0db4f548537823642ec40f4" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46225 } }
I20260618 22:28:17.743829 21516 catalog_manager.cc:5495] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36124b95fa794afd8a07b02242dd6ee8 (127.10.242.195). New cstate: current_term: 1 leader_uuid: "36124b95fa794afd8a07b02242dd6ee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36124b95fa794afd8a07b02242dd6ee8" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a3971e723334ce6b52e0ee48e68a00b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 41369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38bd2d2e0db4f548537823642ec40f4" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46225 } health_report { overall_health: UNKNOWN } } }
I20260618 22:28:17.759893 21987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:17.755068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37158 (local address 127.10.242.195:39685)
0618 22:28:17.755264 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:17.755270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:17.755286 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:28:17.755362 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:17.755366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:17.755443 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:17.755568 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:28:17.755944 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.756631 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.757941 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.758325 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.758430 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:17.758854 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:28:17.759342 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:28:17.759479 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:17.759564 (+ 85us) server_negotiation.cc:299] Negotiation successful
0618 22:28:17.759652 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:28:17.767823 21795 raft_consensus.cc:1261] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b [term 1 FOLLOWER]: Refusing update from remote peer 36124b95fa794afd8a07b02242dd6ee8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:28:17.767835 21661 raft_consensus.cc:1261] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4 [term 1 FOLLOWER]: Refusing update from remote peer 36124b95fa794afd8a07b02242dd6ee8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:28:17.768743 21998 consensus_queue.cc:1035] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38bd2d2e0db4f548537823642ec40f4" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:28:17.769011 22001 consensus_queue.cc:1035] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a3971e723334ce6b52e0ee48e68a00b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 41369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:28:17.782230 22004 mvcc.cc:205] Tried to move back new op lower bound from 7298341674047266816 to 7298341673944805376. Current Snapshot: MvccSnapshot[applied={T|T < 7298341674047266816}]
I20260618 22:28:17.795629 21516 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:28:17.795953 21516 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:28:17.799618 21516 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c6f9e86cbc3544b0b8834023d9623b39: Sending DeleteTablet for 3 replicas of tablet 8aa8a7953cd244cdb76ac856fb8a47cd
I20260618 22:28:17.800809 21641 tablet_service.cc:1514] Processing DeleteTablet for tablet 8aa8a7953cd244cdb76ac856fb8a47cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:28:17 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:53554
I20260618 22:28:17.800815 21775 tablet_service.cc:1514] Processing DeleteTablet for tablet 8aa8a7953cd244cdb76ac856fb8a47cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:28:17 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:60064
I20260618 22:28:17.801543 21908 tablet_service.cc:1514] Processing DeleteTablet for tablet 8aa8a7953cd244cdb76ac856fb8a47cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:28:17 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:37148
I20260618 22:28:17.801760 22013 tablet_replica.cc:330] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4: stopping tablet replica
I20260618 22:28:17.802237 22013 raft_consensus.cc:2229] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:28:17.802817 22013 raft_consensus.cc:2258] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:28:17.802996 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 21578
I20260618 22:28:17.803704 22015 tablet_replica.cc:330] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8: stopping tablet replica
I20260618 22:28:17.805635 22013 ts_tablet_manager.cc:1905] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:28:17.811654 22015 raft_consensus.cc:2229] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:28:17.814334 22014 tablet_replica.cc:330] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b: stopping tablet replica
I20260618 22:28:17.814795 22014 raft_consensus.cc:2229] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:28:17.815183 22015 raft_consensus.cc:2258] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:28:17.815294 22014 raft_consensus.cc:2258] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:28:17.816686 22013 ts_tablet_manager.cc:1918] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:28:17.816913 22013 log.cc:1196] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/8aa8a7953cd244cdb76ac856fb8a47cd
I20260618 22:28:17.817606 22013 ts_tablet_manager.cc:1939] T 8aa8a7953cd244cdb76ac856fb8a47cd P a38bd2d2e0db4f548537823642ec40f4: Deleting consensus metadata
I20260618 22:28:17.817987 22015 ts_tablet_manager.cc:1905] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:28:17.819607 21504 catalog_manager.cc:4841] TS a38bd2d2e0db4f548537823642ec40f4 (127.10.242.193:46225): tablet 8aa8a7953cd244cdb76ac856fb8a47cd (table test-table [id=8a8d04ab9b214c1aa03401f2e03ed8e2]) successfully deleted
I20260618 22:28:17.821137 22014 ts_tablet_manager.cc:1905] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 18 22:28:17 dist-test-slave-0956 krb5kdc[21472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:28:17.826223 22015 ts_tablet_manager.cc:1918] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:28:17.826592 22015 log.cc:1196] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/8aa8a7953cd244cdb76ac856fb8a47cd
I20260618 22:28:17.827186 22015 ts_tablet_manager.cc:1939] T 8aa8a7953cd244cdb76ac856fb8a47cd P 36124b95fa794afd8a07b02242dd6ee8: Deleting consensus metadata
I20260618 22:28:17.828960 22014 ts_tablet_manager.cc:1918] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:28:17.829195 22014 log.cc:1196] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/8aa8a7953cd244cdb76ac856fb8a47cd
I20260618 22:28:17.829696 22014 ts_tablet_manager.cc:1939] T 8aa8a7953cd244cdb76ac856fb8a47cd P 5a3971e723334ce6b52e0ee48e68a00b: Deleting consensus metadata
I20260618 22:28:17.830111 21504 catalog_manager.cc:4841] TS 36124b95fa794afd8a07b02242dd6ee8 (127.10.242.195:39685): tablet 8aa8a7953cd244cdb76ac856fb8a47cd (table test-table [id=8a8d04ab9b214c1aa03401f2e03ed8e2]) successfully deleted
I20260618 22:28:17.831475 21502 catalog_manager.cc:4841] TS 5a3971e723334ce6b52e0ee48e68a00b (127.10.242.194:41369): tablet 8aa8a7953cd244cdb76ac856fb8a47cd (table test-table [id=8a8d04ab9b214c1aa03401f2e03ed8e2]) successfully deleted
I20260618 22:28:17.832696 21990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:17.804618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53564 (local address 127.10.242.193:46225)
0618 22:28:17.804762 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:17.804768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:17.814399 (+ 9631us) server_negotiation.cc:407] Connection header received
0618 22:28:17.814896 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:17.814902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:17.814979 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:17.815059 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:17.816912 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.818131 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.820066 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.820655 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.820747 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:17.826005 (+ 5258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:17.826025 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:17.826044 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:28:17.826090 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:17.829655 (+ 3565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:28:17.830630 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:28:17.830640 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:28:17.830643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:28:17.830802 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:28:17.831167 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:28:17.831172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:28:17.831176 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:28:17.831616 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:17.831915 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:17.832165 (+ 250us) server_negotiation.cc:299] Negotiation successful
0618 22:28:17.832368 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260618 22:28:17.833405 21701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:28:17.920701 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 21578
I20260618 22:28:17.941787 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 21712
Jun 18 22:28:17 dist-test-slave-0956 krb5kdc[21472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.194@KRBTEST.COM
I20260618 22:28:17.958376 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:17.942678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60068 (local address 127.10.242.194:41369)
0618 22:28:17.942846 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:17.942850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:17.942876 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:28:17.943063 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:17.943073 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:17.943145 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:17.943233 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:17.944765 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.945609 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.947414 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:17.947807 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:17.947900 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:17.952803 (+ 4903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:17.952819 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:17.952835 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:28:17.952876 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:17.955695 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:28:17.956525 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:28:17.956532 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:28:17.956534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:28:17.956655 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:28:17.957039 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:28:17.957043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:28:17.957045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:28:17.957455 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:17.957662 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:17.957979 (+ 317us) server_negotiation.cc:299] Negotiation successful
0618 22:28:17.958131 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:28:17.959023 21835 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:28:18.077910 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 21712
I20260618 22:28:18.096788 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 21845
Jun 18 22:28:18 dist-test-slave-0956 krb5kdc[21472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.195@KRBTEST.COM
I20260618 22:28:18.114712 21987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:28:18.097742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37166 (local address 127.10.242.195:39685)
0618 22:28:18.097914 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0618 22:28:18.097918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:28:18.097939 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:28:18.098148 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:28:18.098152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:28:18.098254 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:28:18.098346 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:28:18.099966 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:18.100786 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:18.102550 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:28:18.103028 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:28:18.103130 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:28:18.108501 (+ 5371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:28:18.108514 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:28:18.108529 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:28:18.108573 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:28:18.111721 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:28:18.112598 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:28:18.112606 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:28:18.112611 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:28:18.112772 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:28:18.113235 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:28:18.113239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:28:18.113241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:28:18.113619 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:28:18.113854 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0618 22:28:18.114199 (+ 345us) server_negotiation.cc:299] Negotiation successful
0618 22:28:18.114366 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:28:18.115470 21968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:28:18.234616 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 21845
I20260618 22:28:18.249397 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 21485
I20260618 22:28:18.250501 21547 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:28:18.357048 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 21485
2026-06-18T22:28:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4034 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:28:18 dist-test-slave-0956 krb5kdc[22023](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:28:18 dist-test-slave-0956 krb5kdc[22023](info): set up 2 sockets
Jun 18 22:28:18 dist-test-slave-0956 krb5kdc[22023](info): commencing operation
krb5kdc: starting...
W20260618 22:28:19.386021 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:28:19 dist-test-slave-0956 krb5kdc[22023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:28:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:28:19Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.242.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.242.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.242.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/rangeradmin_127.10.242.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.242.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/rangeradmin_127.10.242.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.242.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.242.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.242.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/rangerlookup_127.10.242.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.242.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/rangerlookup_127.10.242.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.242.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.242.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.242.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/HTTP_127.10.242.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.242.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/HTTP_127.10.242.212@KRBTEST.COM.keytab.
I20260618 22:28:19.695737 11211 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJpXzaW/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/postgres -l logfile start
2026-06-18 22:28:22.028 UTC [22053] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-18 22:28:22.028 UTC [22053] LOG: listening on IPv4 address "127.10.242.212", port 58525
2026-06-18 22:28:22.031 UTC [22053] LOG: listening on Unix socket "/tmp/.s.PGSQL.58525"
2026-06-18 22:28:22.050 UTC [22056] LOG: database system was shut down at 2026-06-18 22:28:20 UTC
2026-06-18 22:28:22.054 UTC [22053] LOG: database system is ready to accept connections
I20260618 22:28:22.945426 11211 mini_postgres.cc:96] Postgres bound to 58525
2026-06-18 22:28:22.961 UTC [22064] FATAL: database "slave" does not exist
127.10.242.212:58525 - accepting connections
I20260618 22:28:22.962601 11211 mini_ranger.cc:161] Starting Ranger...
I20260618 22:28:22.987411 11211 mini_ranger.cc:84] Created miniranger Postgres user
I20260618 22:28:23.452311 11211 mini_ranger.cc:87] Created ranger Postgres database
I20260618 22:28:23.452464 11211 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ranger-admin
2026-06-18 22:28:23,733 [I] DB FLAVOR :POSTGRES
2026-06-18 22:28:23,734 [I] --------- Verifying Ranger DB connection ---------
2026-06-18 22:28:23,734 [I] Checking connection..
2026-06-18 22:28:23,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-18 22:28:23,987 [I] Checking connection passed.
2026-06-18 22:28:23,987 [I] --------- Verifying version history table ---------
2026-06-18 22:28:23,988 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-18 22:28:24,248 [I] Table x_db_version_h does not exist in database ranger
2026-06-18 22:28:24,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-18 22:28:24,517 [I] Table x_db_version_h does not exist in database ranger
2026-06-18 22:28:24,517 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-18 22:28:24,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-18 22:28:24.759 UTC [22128] WARNING: there is no transaction in progress
2026-06-18 22:28:24,765 [I] create_dbversion_catalog.sql file imported successfully
2026-06-18 22:28:24,765 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-18 22:28:25,026 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-18 22:28:25,026 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-18 22:28:25,026 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-18 22:28:25,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-18 22:28:25,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-0956.c.gcp-upstream.internal','N') ;"
2026-06-18 22:28:25,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-18 22:28:25,997 [I] Table x_portal_user does not exist in database ranger
2026-06-18 22:28:25,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-18 22:28:26,241 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-18 22:28:26,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-18 22:28:26,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-18 22:28:26,734 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-18 22:28:26,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-18 22:28:27.690 UTC [22236] WARNING: there is no transaction in progress
2026-06-18 22:28:27.708 UTC [22236] WARNING: there is no transaction in progress
2026-06-18 22:28:27.725 UTC [22236] WARNING: there is no transaction in progress
2026-06-18 22:28:27.743 UTC [22236] WARNING: there is no transaction in progress
2026-06-18 22:28:27.763 UTC [22236] WARNING: there is no transaction in progress
2026-06-18 22:28:27.783 UTC [22236] WARNING: there is no transaction in progress
2026-06-18 22:28:27.851 UTC [22236] WARNING: there is no transaction in progress
2026-06-18 22:28:27.859 UTC [22236] WARNING: there is no transaction in progress
2026-06-18 22:28:27.875 UTC [22236] WARNING: there is no transaction in progress
2026-06-18 22:28:27.885 UTC [22236] WARNING: there is no transaction in progress
2026-06-18 22:28:27.896 UTC [22236] WARNING: there is no transaction in progress
2026-06-18 22:28:27.911 UTC [22236] WARNING: there is no transaction in progress
2026-06-18 22:28:27.922 UTC [22236] WARNING: there is no transaction in progress
2026-06-18 22:28:27.929 UTC [22236] WARNING: there is no transaction in progress
2026-06-18 22:28:27.937 UTC [22236] WARNING: there is no transaction in progress
2026-06-18 22:28:27.944 UTC [22236] WARNING: there is no transaction in progress
2026-06-18 22:28:28,559 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-18 22:28:28,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-18 22:28:28,812 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-18 22:28:28,812 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-18 22:28:29,074 [I] Table x_portal_user already exists in database 'ranger'
2026-06-18 22:28:29,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-18 22:28:29,310 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-18 22:28:29,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-18 22:28:29,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-18 22:28:29,775 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-0956.c.gcp-upstream.internal';"
2026-06-18 22:28:30,023 [I] CORE_DB_SCHEMA import status has been updated
2026-06-18 22:28:30,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-18 22:28:30,278 [I] DB_PATCHES have already been applied
I20260618 22:28:30.284485 11211 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJpXzaW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260618 22:28:30.284597 11211 mini_ranger.cc:193] Using host: 127.10.242.212
I20260618 22:28:30.298473 11211 mini_ranger.cc:227] Ranger admin URL: http://127.10.242.212:59751
Jun 18, 2026 10:28:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 18, 2026 10:28:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 18, 2026 10:28:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ranger-admin/ews/webapp
Jun 18, 2026 10:28:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 18, 2026 10:28:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 18, 2026 10:28:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ranger-admin/ews/webapp].
Jun 18, 2026 10:28:31 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260618 22:28:31.331055 11211 mini_ranger.cc:160] Time spent starting Ranger: real 8.368s user 0.005s sys 0.035s
Jun 18, 2026 10:28:51 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 18, 2026 10:28:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Jun 18, 2026 10:28:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 18, 2026 10:28:51 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 18, 2026 10:28:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 18, 2026 10:28:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 18, 2026 10:28:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 18, 2026 10:28:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 18, 2026 10:28:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 18, 2026 10:28:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 18, 2026 10:28:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 18, 2026 10:28:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 18, 2026 10:28:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 18, 2026 10:28:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 18, 2026 10:28:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 18, 2026 10:28:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 18, 2026 10:28:51 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 18, 2026 10:28:52 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260618 22:28:53.363042 11211 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.242.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.242.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.242.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/rangerkms_127.10.242.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.242.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/rangerkms_127.10.242.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.242.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.242.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.242.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/HTTP_127.10.242.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.242.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/HTTP_127.10.242.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 18 22:28:53 dist-test-slave-0956 krb5kdc[22023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821733, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260618 22:28:53.513665 11211 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260618 22:28:53.536960 11211 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260618 22:28:54.329780 11211 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260618 22:28:54.329901 11211 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ranger-kms
I20260618 22:28:54.330034 11211 mini_ranger_kms.cc:227] Using postgres at 127.10.242.212:58525
2026-06-18 22:28:54,706 [I] DB FLAVOR :POSTGRES
2026-06-18 22:28:54,707 [I] --------- Verifying Ranger DB connection ---------
2026-06-18 22:28:54,707 [I] Checking connection
2026-06-18 22:28:54,707 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-18 22:28:54,948 [I] connection success
2026-06-18 22:28:54,949 [I] --------- Verifying Ranger DB tables ---------
2026-06-18 22:28:54,949 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-18 22:28:54,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-18 22:28:55,190 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-18 22:28:55,190 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-18 22:28:55,191 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-18 22:28:55,191 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:58525/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-18 22:28:55,444 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260618 22:28:55.898509 11211 mini_ranger_kms.cc:310] Created kms service
I20260618 22:28:56.069456 11211 mini_ranger_kms.cc:326] Created kudu user
I20260618 22:28:56.112352 11211 mini_ranger_kms.cc:343] Created rangerkms user
Jun 18 22:28:56 dist-test-slave-0956 krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821733, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
I20260618 22:28:56.527791 11211 mini_ranger_kms.cc:384] Added ranger policy
I20260618 22:28:56.528092 11211 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260618 22:28:56.528141 11211 mini_ranger_kms.cc:240] Using host: 127.10.242.212
I20260618 22:28:56.547461 11211 mini_ranger_kms.cc:276] Ranger KMS PID: 22416
I20260618 22:28:56.547665 11211 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.242.212:48713
Jun 18, 2026 10:28:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 18, 2026 10:28:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 18, 2026 10:28:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 18, 2026 10:28:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 18, 2026 10:28:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ranger-kms/ews/webapp].
Jun 18, 2026 10:28:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Jun 18 22:28:58 dist-test-slave-0956 krb5kdc[22023](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781821738, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.242.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:28:58.697340 11211 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.184s user 0.100s sys 0.308s
I20260618 22:28:58.697597 11211 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260618 22:28:58.697814 11211 mini_ranger_kms.cc:401] 127.10.242.212:48713/kms/v1/keys
Jun 18 22:28:59 dist-test-slave-0956 krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821738, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.242.212@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
Jun 18 22:28:59 dist-test-slave-0956 krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821738, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.242.212@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
Jun 18 22:29:01 dist-test-slave-0956 krb5kdc[22023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:29:01.781869 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:45615
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:45615
--ranger_config_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.212:48713/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:29:02.144798 22467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:29:02.145107 22467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:29:02.145218 22467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:29:02.152904 22467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:29:02.153015 22467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:29:02.153069 22467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:29:02.153162 22467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:29:02.153220 22467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:29:02.163239 22467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.242.212:48713/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:45615
--ranger_config_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:45615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:29:02.164521 22467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:29:02.165853 22467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:29:02.177563 22472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:02.177498 22475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:02.177536 22473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:29:02.178505 22467 server_base.cc:1034] running on GCE node
I20260618 22:29:02.179307 22467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:29:02.180691 22467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:29:02.181887 22467 hybrid_clock.cc:648] HybridClock initialized: now 1781821742181836 us; error 46 us; skew 500 ppm
I20260618 22:29:02.182255 22467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:29:02 dist-test-slave-0956 krb5kdc[22023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821742, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:29:02.185221 22467 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:29:02.186615 22467 webserver.cc:460] Webserver started at http://127.10.242.254:39037/ using document root <none> and password file <none>
I20260618 22:29:02.187233 22467 fs_manager.cc:359] Metadata directory not provided
I20260618 22:29:02.187322 22467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:29:02.187562 22467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:29:02 dist-test-slave-0956 krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821742, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
I20260618 22:29:02.246281 22467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "fa1d00a790eb408d92751b416c131b16"
format_stamp: "Formatted at 2026-06-18 22:29:02 on dist-test-slave-0956"
server_key: "bd08f6a85294b354aa769fb8a6eba335"
server_key_iv: "484e11a1dd4df931c2506e689cf9fc86"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:02.247366 22467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "fa1d00a790eb408d92751b416c131b16"
format_stamp: "Formatted at 2026-06-18 22:29:02 on dist-test-slave-0956"
server_key: "bd08f6a85294b354aa769fb8a6eba335"
server_key_iv: "484e11a1dd4df931c2506e689cf9fc86"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:02.254350 22467 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260618 22:29:02.274003 22482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:02.275723 22467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:29:02.275906 22467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "fa1d00a790eb408d92751b416c131b16"
format_stamp: "Formatted at 2026-06-18 22:29:02 on dist-test-slave-0956"
server_key: "bd08f6a85294b354aa769fb8a6eba335"
server_key_iv: "484e11a1dd4df931c2506e689cf9fc86"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:02.276156 22467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:29:02.291998 22467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:29:02.296099 22467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:29:02.296396 22467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:29:02.317883 22467 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:45615
I20260618 22:29:02.317900 22533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:45615 every 8 connection(s)
I20260618 22:29:02.319290 22467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:29:02.323406 22534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:29:02.325600 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 22467
I20260618 22:29:02.325771 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:29:02.326377 11211 external_mini_cluster.cc:1428] Setting key 9722dc8278be997e805cb5928cc1891f
I20260618 22:29:02.336078 22534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16: Bootstrap starting.
I20260618 22:29:02.339398 22534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16: Neither blocks nor log segments found. Creating new log.
I20260618 22:29:02.340345 22534 log.cc:824] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:29:02 dist-test-slave-0956 krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:29:02.343768 22534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16: No bootstrap required, opened a new log
I20260618 22:29:02.349989 22537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:02.328368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41600 (local address 127.10.242.254:45615)
0618 22:29:02.329129 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:02.329144 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:02.329192 (+ 48us) server_negotiation.cc:407] Connection header received
0618 22:29:02.331374 (+ 2182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:02.331404 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:02.331929 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:02.332371 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:02.333489 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:02.335392 (+ 1903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:02.336670 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:02.337678 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:02.337858 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:02.341575 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:02.341584 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:02.341614 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:02.341665 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:02.345235 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:02.345988 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:02.345998 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:02.346006 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:02.346197 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:02.346525 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:02.346530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:02.346534 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:02.347247 (+ 713us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:02.347527 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:02.347906 (+ 379us) server_negotiation.cc:299] Negotiation successful
0618 22:29:02.348316 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":288,"threads_started":1}
I20260618 22:29:02.353215 22534 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1d00a790eb408d92751b416c131b16" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45615 } }
I20260618 22:29:02.353542 22534 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:29:02.353612 22534 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa1d00a790eb408d92751b416c131b16, State: Initialized, Role: FOLLOWER
I20260618 22:29:02.354076 22534 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1d00a790eb408d92751b416c131b16" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45615 } }
I20260618 22:29:02.354272 22534 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:29:02.354359 22534 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:29:02.354477 22534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:29:02.356552 22534 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1d00a790eb408d92751b416c131b16" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45615 } }
I20260618 22:29:02.356981 22534 leader_election.cc:302] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa1d00a790eb408d92751b416c131b16; no voters:
I20260618 22:29:02.357523 22534 leader_election.cc:288] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:29:02.357649 22539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:29:02.359091 22539 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16 [term 1 LEADER]: Becoming Leader. State: Replica: fa1d00a790eb408d92751b416c131b16, State: Running, Role: LEADER
I20260618 22:29:02.359753 22534 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:29:02.359732 22539 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1d00a790eb408d92751b416c131b16" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45615 } }
I20260618 22:29:02.364542 22541 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa1d00a790eb408d92751b416c131b16. Latest consensus state: current_term: 1 leader_uuid: "fa1d00a790eb408d92751b416c131b16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1d00a790eb408d92751b416c131b16" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45615 } } }
I20260618 22:29:02.364993 22541 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16 [sys.catalog]: This master's current role is: LEADER
I20260618 22:29:02.364509 22540 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa1d00a790eb408d92751b416c131b16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1d00a790eb408d92751b416c131b16" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45615 } } }
I20260618 22:29:02.365912 22540 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16 [sys.catalog]: This master's current role is: LEADER
I20260618 22:29:02.367384 22549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:29:02.372244 22549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:29:02.372927 22534 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260618 22:29:02.395525 22549 catalog_manager.cc:1347] Generated new cluster ID: 4c66bdb2454340908d39aa4c01bb54e9
I20260618 22:29:02.395906 22549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:29:02.413168 22549 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:29:02.415241 22549 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:29:02.426079 22549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16: Generated new TSK 0
I20260618 22:29:02.427167 22549 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
I20260618 22:29:03.163270 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.212:48713/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45615
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:29:03 dist-test-slave-0956 krb5kdc[22023](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781821743, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260618 22:29:03.590709 22579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:29:03.591029 22579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:29:03.591197 22579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:29:03.599637 22579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:29:03.599766 22579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:29:03.599974 22579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:29:03.610877 22579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.242.212:48713/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:29:03.612330 22579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:29:03.614006 22579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:29:03.630910 22589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:03.631990 22590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:03.640259 22592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:29:03.641505 22579 server_base.cc:1034] running on GCE node
I20260618 22:29:03.642251 22579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:29:03.643319 22579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:29:03.644644 22579 hybrid_clock.cc:648] HybridClock initialized: now 1781821743644575 us; error 63 us; skew 500 ppm
I20260618 22:29:03.645005 22579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:29:03 dist-test-slave-0956 krb5kdc[22023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821743, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:29:03.649161 22579 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:29:03.660208 22579 webserver.cc:460] Webserver started at http://127.10.242.193:35693/ using document root <none> and password file <none>
I20260618 22:29:03.661103 22579 fs_manager.cc:359] Metadata directory not provided
I20260618 22:29:03.661260 22579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:29:03.661602 22579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:29:03 dist-test-slave-0956 krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821743, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
I20260618 22:29:03.686808 22579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "941a6b38cb1843c3918e7c61a779d8d3"
format_stamp: "Formatted at 2026-06-18 22:29:03 on dist-test-slave-0956"
server_key: "5a7d72cf87018586de6a51167aea2b25"
server_key_iv: "8a3f219029209e266037c86ec55a00e0"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:03.687660 22579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "941a6b38cb1843c3918e7c61a779d8d3"
format_stamp: "Formatted at 2026-06-18 22:29:03 on dist-test-slave-0956"
server_key: "5a7d72cf87018586de6a51167aea2b25"
server_key_iv: "8a3f219029209e266037c86ec55a00e0"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:03.693236 22579 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260618 22:29:03.729818 22599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:03.733246 22579 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.003s sys 0.006s
I20260618 22:29:03.733484 22579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "941a6b38cb1843c3918e7c61a779d8d3"
format_stamp: "Formatted at 2026-06-18 22:29:03 on dist-test-slave-0956"
server_key: "5a7d72cf87018586de6a51167aea2b25"
server_key_iv: "8a3f219029209e266037c86ec55a00e0"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:03.733788 22579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:29:03.782128 22579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:29:03.787137 22579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:29:03.787436 22579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:29:03.788425 22579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:29:03.789891 22579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:29:03.789974 22579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:03.790128 22579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:29:03.790181 22579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:03.895774 22579 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:35637
I20260618 22:29:03.895776 22711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:35637 every 8 connection(s)
I20260618 22:29:03.897207 22579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:29:03.906495 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 22579
I20260618 22:29:03.906723 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:29:03.907490 11211 external_mini_cluster.cc:1428] Setting key 705758e5ad2bafacf4407b3c50c0010f
Jun 18 22:29:03 dist-test-slave-0956 krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821743, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:29:03.934324 22715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:03.902443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45615 (local address 127.10.242.193:38487)
0618 22:29:03.905701 (+ 3258us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:03.905749 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:03.908241 (+ 2492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:03.909633 (+ 1392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:03.909647 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:03.910276 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:29:03.910957 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:03.910978 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:03.912579 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:03.912584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:03.914601 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:03.914628 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:03.916176 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:03.916185 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:03.916498 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:03.921290 (+ 4792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:29:03.921324 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:29:03.924919 (+ 3595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:29:03.929272 (+ 4353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:29:03.929283 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:29:03.929303 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:29:03.929820 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:29:03.930344 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:29:03.930349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:29:03.930353 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:29:03.930647 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:29:03.931383 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:29:03.931392 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:29:03.931917 (+ 525us) client_negotiation.cc:769] Sending connection context
0618 22:29:03.932240 (+ 323us) client_negotiation.cc:240] Negotiation successful
0618 22:29:03.932814 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2948,"thread_start_us":2707,"threads_started":1}
I20260618 22:29:03.935524 22714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:03.901455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:38487 (local address 127.10.242.254:45615)
0618 22:29:03.908890 (+ 7435us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:03.908895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:03.908913 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:29:03.908964 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:03.908969 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:03.909054 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:03.909185 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:03.911153 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:03.912362 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:03.914794 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:03.915437 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:03.915564 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:03.925296 (+ 9732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:03.925305 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:03.925309 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:03.925356 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:03.929109 (+ 3753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:03.929998 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:03.930003 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:03.930008 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:03.930176 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:03.930791 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:03.930796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:03.930800 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:03.931156 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:03.931300 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:03.934890 (+ 3590us) server_negotiation.cc:299] Negotiation successful
0618 22:29:03.935160 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7322,"thread_start_us":8378,"threads_started":1}
WARNING: no policy specified for kudu/127.10.242.194@KRBTEST.COM; defaulting to no policy
I20260618 22:29:03.937848 22712 heartbeater.cc:347] Connected to a master server at 127.10.242.254:45615
I20260618 22:29:03.938190 22712 heartbeater.cc:464] Registering TS with master...
I20260618 22:29:03.939139 22712 heartbeater.cc:511] Master 127.10.242.254:45615 requested a full tablet report, sending...
I20260618 22:29:03.941923 22499 ts_manager.cc:194] Registered new tserver with Master: 941a6b38cb1843c3918e7c61a779d8d3 (127.10.242.193:35637)
I20260618 22:29:03.944002 22499 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:38487
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.194@KRBTEST.COM" created.
Jun 18 22:29:03 dist-test-slave-0956 krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821743, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
I20260618 22:29:03.987033 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.212:48713/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45615
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:29:04 dist-test-slave-0956 krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821743, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
W20260618 22:29:04.461366 22718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:29:04.461869 22718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:29:04.462538 22718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:29:04.474022 22718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:29:04.474275 22718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:29:04.474597 22718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:29:04.490764 22718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.242.212:48713/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:29:04.492475 22718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:29:04.494148 22718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:29:04.530778 22726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:04.538971 22729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:29:04.539428 22718 server_base.cc:1034] running on GCE node
W20260618 22:29:04.539916 22725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:29:04.540798 22718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:29:04.550380 22718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:29:04.551924 22718 hybrid_clock.cc:648] HybridClock initialized: now 1781821744551807 us; error 113 us; skew 500 ppm
I20260618 22:29:04.552443 22718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:29:04 dist-test-slave-0956 krb5kdc[22023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821744, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:29:04.556950 22718 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:29:04.577768 22718 webserver.cc:460] Webserver started at http://127.10.242.194:40939/ using document root <none> and password file <none>
I20260618 22:29:04.578900 22718 fs_manager.cc:359] Metadata directory not provided
I20260618 22:29:04.579186 22718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:29:04.579649 22718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:29:04 dist-test-slave-0956 krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821744, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
I20260618 22:29:04.618350 22718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "fa730adf1dbb4bce9a9de421623a69de"
format_stamp: "Formatted at 2026-06-18 22:29:04 on dist-test-slave-0956"
server_key: "9317d9555135cc53444f9006c45ded36"
server_key_iv: "69db93c09de7f7a361edee7722fbd28a"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:04.626184 22718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "fa730adf1dbb4bce9a9de421623a69de"
format_stamp: "Formatted at 2026-06-18 22:29:04 on dist-test-slave-0956"
server_key: "9317d9555135cc53444f9006c45ded36"
server_key_iv: "69db93c09de7f7a361edee7722fbd28a"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:04.650442 22718 fs_manager.cc:691] Time spent creating directory manager: real 0.024s user 0.003s sys 0.006s
I20260618 22:29:04.691823 22735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:04.693969 22718 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.005s sys 0.000s
I20260618 22:29:04.694257 22718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "fa730adf1dbb4bce9a9de421623a69de"
format_stamp: "Formatted at 2026-06-18 22:29:04 on dist-test-slave-0956"
server_key: "9317d9555135cc53444f9006c45ded36"
server_key_iv: "69db93c09de7f7a361edee7722fbd28a"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:04.694592 22718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:29:04.754127 22718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:29:04.759297 22718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:29:04.759748 22718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:29:04.767150 22718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:29:04.775254 22718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:29:04.775440 22718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260618 22:29:04.775605 22718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:29:04.775698 22718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:04.938568 22718 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:36829
I20260618 22:29:04.938591 22854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:36829 every 8 connection(s)
I20260618 22:29:04.940038 22718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:29:04.948707 22712 heartbeater.cc:503] Master 127.10.242.254:45615 was elected leader, sending a full tablet report...
I20260618 22:29:04.949456 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 22718
I20260618 22:29:04.949653 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:29:04.950254 11211 external_mini_cluster.cc:1428] Setting key b93df37f7b1fe6796e65ba2cee77c71c
Jun 18 22:29:04 dist-test-slave-0956 krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821744, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:29:04.971066 22858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:04.945055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:52337 (local address 127.10.242.254:45615)
0618 22:29:04.950469 (+ 5414us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:04.950475 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:04.950497 (+ 22us) server_negotiation.cc:407] Connection header received
0618 22:29:04.950557 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:04.950563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:04.950656 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:04.950757 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:04.955257 (+ 4500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:04.956460 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:04.958740 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:04.959292 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:04.959376 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:04.963923 (+ 4547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:04.963948 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:04.963960 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:04.964015 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:04.967346 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:04.968343 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:04.968349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:04.968353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:04.968513 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:04.969055 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:04.969060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:04.969063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:04.969435 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:04.969565 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:04.970500 (+ 935us) server_negotiation.cc:299] Negotiation successful
0618 22:29:04.970717 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5284,"thread_start_us":6085,"threads_started":1}
I20260618 22:29:04.973701 22857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:04.943855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45615 (local address 127.10.242.194:52337)
0618 22:29:04.946236 (+ 2381us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:04.946299 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:04.948724 (+ 2425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:04.953576 (+ 4852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:04.953590 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:04.954188 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:29:04.955012 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:04.955038 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:04.956669 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:04.956674 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:04.958571 (+ 1897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:04.958598 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:04.959681 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:04.959687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:04.960019 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:04.960758 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:29:04.960786 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:29:04.963415 (+ 2629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:29:04.967564 (+ 4149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:29:04.967573 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:29:04.967593 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:29:04.968173 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:29:04.968657 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:29:04.968662 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:29:04.968666 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:29:04.968929 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:29:04.969650 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:29:04.969659 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:29:04.970109 (+ 450us) client_negotiation.cc:769] Sending connection context
0618 22:29:04.971685 (+ 1576us) client_negotiation.cc:240] Negotiation successful
0618 22:29:04.972255 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2067,"thread_start_us":1834,"threads_started":1}
I20260618 22:29:04.975761 22855 heartbeater.cc:347] Connected to a master server at 127.10.242.254:45615
I20260618 22:29:04.976080 22855 heartbeater.cc:464] Registering TS with master...
I20260618 22:29:04.976946 22855 heartbeater.cc:511] Master 127.10.242.254:45615 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.242.195@KRBTEST.COM; defaulting to no policy
I20260618 22:29:04.978765 22498 ts_manager.cc:194] Registered new tserver with Master: fa730adf1dbb4bce9a9de421623a69de (127.10.242.194:36829)
I20260618 22:29:04.979725 22498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:52337
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
I20260618 22:29:05.007462 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.212:48713/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45615
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:29:05.340322 22861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:29:05.340612 22861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:29:05.340754 22861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:29:05.347839 22861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:29:05.347958 22861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:29:05.348105 22861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:29:05.357889 22861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.242.212:48713/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:29:05.359180 22861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:29:05.360497 22861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:29:05.374910 22869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:05.374938 22867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:05.375397 22866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:29:05.375535 22861 server_base.cc:1034] running on GCE node
I20260618 22:29:05.376156 22861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:29:05.377143 22861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:29:05.378373 22861 hybrid_clock.cc:648] HybridClock initialized: now 1781821745378325 us; error 30 us; skew 500 ppm
I20260618 22:29:05.378757 22861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:29:05 dist-test-slave-0956 krb5kdc[22023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821745, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:29:05.386633 22861 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:29:05.388229 22861 webserver.cc:460] Webserver started at http://127.10.242.195:37401/ using document root <none> and password file <none>
I20260618 22:29:05.388922 22861 fs_manager.cc:359] Metadata directory not provided
I20260618 22:29:05.389045 22861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:29:05.389376 22861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:29:05 dist-test-slave-0956 krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821745, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
I20260618 22:29:05.406800 22861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "6315f30a2f35495ba4707f62e51e6a77"
format_stamp: "Formatted at 2026-06-18 22:29:05 on dist-test-slave-0956"
server_key: "bea3e10c1cb835a955b709d864f38df8"
server_key_iv: "190eab7ce35c6581f8374675c6c1f2a6"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:05.407648 22861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "6315f30a2f35495ba4707f62e51e6a77"
format_stamp: "Formatted at 2026-06-18 22:29:05 on dist-test-slave-0956"
server_key: "bea3e10c1cb835a955b709d864f38df8"
server_key_iv: "190eab7ce35c6581f8374675c6c1f2a6"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:05.413314 22861 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260618 22:29:05.431241 22876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:05.432812 22861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260618 22:29:05.432971 22861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "6315f30a2f35495ba4707f62e51e6a77"
format_stamp: "Formatted at 2026-06-18 22:29:05 on dist-test-slave-0956"
server_key: "bea3e10c1cb835a955b709d864f38df8"
server_key_iv: "190eab7ce35c6581f8374675c6c1f2a6"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:05.433233 22861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:29:05.461045 22861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:29:05.465287 22861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:29:05.465569 22861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:29:05.466550 22861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:29:05.467938 22861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:29:05.468010 22861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:05.468125 22861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:29:05.468173 22861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:05.500844 22861 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:41693
I20260618 22:29:05.500883 22988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:41693 every 8 connection(s)
I20260618 22:29:05.502141 22861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:29:05.508635 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 22861
I20260618 22:29:05.508796 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:29:05.509266 11211 external_mini_cluster.cc:1428] Setting key 9489cb2636921f837f9d23f24ed9a7d2
Jun 18 22:29:05 dist-test-slave-0956 krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821745, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:29:05.521905 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.504668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:55559 (local address 127.10.242.254:45615)
0618 22:29:05.505136 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:05.505142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:05.506694 (+ 1552us) server_negotiation.cc:407] Connection header received
0618 22:29:05.508577 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:05.508584 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:05.508671 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:05.508779 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:05.510539 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.511351 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.513005 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.513502 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.513624 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.516854 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:05.516859 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:05.516862 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:05.516895 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:05.519337 (+ 2442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:05.520002 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:05.520006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:05.520008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:05.520146 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:05.520491 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:05.520494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:05.520496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:05.520737 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:05.520836 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:05.521502 (+ 666us) server_negotiation.cc:299] Negotiation successful
0618 22:29:05.521656 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":195,"threads_started":1}
I20260618 22:29:05.522962 22992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.505340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45615 (local address 127.10.242.195:55559)
0618 22:29:05.506512 (+ 1172us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:05.506564 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:05.508244 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:05.509032 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:05.509041 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:05.509563 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:29:05.510282 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:05.510310 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.511512 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.511516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:05.512847 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:05.512861 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.513722 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.513730 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:05.513973 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.514614 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:29:05.514637 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:29:05.516633 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:29:05.519466 (+ 2833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:29:05.519473 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:29:05.519487 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:29:05.519913 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:29:05.520222 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:29:05.520225 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:29:05.520227 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:29:05.520415 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:29:05.520822 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:29:05.520828 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:29:05.521190 (+ 362us) client_negotiation.cc:769] Sending connection context
0618 22:29:05.521417 (+ 227us) client_negotiation.cc:240] Negotiation successful
0618 22:29:05.521807 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"thread_start_us":697,"threads_started":1}
I20260618 22:29:05.524251 22989 heartbeater.cc:347] Connected to a master server at 127.10.242.254:45615
I20260618 22:29:05.524528 22989 heartbeater.cc:464] Registering TS with master...
I20260618 22:29:05.525254 22989 heartbeater.cc:511] Master 127.10.242.254:45615 requested a full tablet report, sending...
I20260618 22:29:05.526484 22498 ts_manager.cc:194] Registered new tserver with Master: 6315f30a2f35495ba4707f62e51e6a77 (127.10.242.195:41693)
I20260618 22:29:05.527277 22498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:55559
I20260618 22:29:05.537217 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:29:05.553928 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.541763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41610 (local address 127.10.242.254:45615)
0618 22:29:05.541933 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:05.541938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:05.542402 (+ 464us) server_negotiation.cc:407] Connection header received
0618 22:29:05.542533 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:05.542536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:05.542617 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:05.542700 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:05.543997 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.544746 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.546167 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.546577 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.546666 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.549099 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:05.549106 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:05.549109 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:05.549144 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:05.551572 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:05.552211 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:05.552215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:05.552217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:05.552332 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:05.552624 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:05.552627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:05.552629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:05.552874 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:05.553064 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:05.553452 (+ 388us) server_negotiation.cc:299] Negotiation successful
0618 22:29:05.553631 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:29:05.558108 22498 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:29:05.565637 22498 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:29:05.598510 23004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.578009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50186 (local address 127.10.242.193:35637)
0618 22:29:05.590719 (+ 12710us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:05.590727 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:05.590753 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:29:05.590896 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:05.590903 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:05.591118 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:05.591300 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:29:05.592274 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.593723 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.596005 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.597466 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.597636 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.597808 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:05.597999 (+ 191us) server_negotiation.cc:299] Negotiation successful
0618 22:29:05.598165 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12525,"thread_start_us":12356,"threads_started":1}
I20260618 22:29:05.598675 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.577793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:35637 (local address 127.0.0.1:50186)
0618 22:29:05.588153 (+ 10360us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:05.588204 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:05.588404 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:05.591424 (+ 3020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:05.591428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:05.591452 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:29:05.592097 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:05.592111 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.593853 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.593857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:05.595841 (+ 1984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:05.595856 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.597664 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.597670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:05.597826 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.597856 (+ 30us) client_negotiation.cc:769] Sending connection context
0618 22:29:05.597952 (+ 96us) client_negotiation.cc:240] Negotiation successful
0618 22:29:05.598218 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10064,"mutex_wait_us":113,"thread_start_us":12790,"threads_started":1}
I20260618 22:29:05.598788 23005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.578217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36386 (local address 127.10.242.194:36829)
0618 22:29:05.587534 (+ 9317us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:05.587547 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:05.589925 (+ 2378us) server_negotiation.cc:407] Connection header received
0618 22:29:05.590075 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:05.590149 (+ 74us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:05.590449 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:05.590856 (+ 407us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:29:05.591549 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.592944 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.595576 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.596998 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.597194 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.597381 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:05.598332 (+ 951us) server_negotiation.cc:299] Negotiation successful
0618 22:29:05.598499 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8587,"thread_start_us":8487,"threads_started":1}
I20260618 22:29:05.599189 23003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.577930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:36829 (local address 127.0.0.1:36386)
0618 22:29:05.589807 (+ 11877us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:05.589826 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:05.589943 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:05.590871 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:05.590877 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:05.590903 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:29:05.591342 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:05.591355 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.593108 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.593114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:05.595419 (+ 2305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:05.595433 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.597511 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.597517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:05.597713 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.597763 (+ 50us) client_negotiation.cc:769] Sending connection context
0618 22:29:05.597889 (+ 126us) client_negotiation.cc:240] Negotiation successful
0618 22:29:05.598199 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11760,"mutex_wait_us":4,"thread_start_us":11639,"threads_started":1}
I20260618 22:29:05.599740 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.580255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49638 (local address 127.10.242.195:41693)
0618 22:29:05.590222 (+ 9967us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:05.590230 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:05.590646 (+ 416us) server_negotiation.cc:407] Connection header received
0618 22:29:05.590739 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:05.590744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:05.590951 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:05.591160 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:29:05.591873 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.593484 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.596080 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.597515 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.597697 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.597857 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:05.599314 (+ 1457us) server_negotiation.cc:299] Negotiation successful
0618 22:29:05.599468 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9806,"thread_start_us":9654,"threads_started":1}
I20260618 22:29:05.600018 23002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.579651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:41693 (local address 127.0.0.1:49638)
0618 22:29:05.590424 (+ 10773us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:05.590441 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:05.590544 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:05.591153 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:05.591158 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:05.591185 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:29:05.591626 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:05.591639 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.593631 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.593635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:05.595942 (+ 2307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:05.595955 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.597979 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.597985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:05.598161 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.598188 (+ 27us) client_negotiation.cc:769] Sending connection context
0618 22:29:05.598272 (+ 84us) client_negotiation.cc:240] Negotiation successful
0618 22:29:05.598392 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10656,"mutex_wait_us":22,"thread_start_us":8083,"threads_started":1}
I20260618 22:29:05.602933 22647 tablet_service.cc:1467] Processing CreateTablet for tablet a2cc56ba98d149429f05d6635e090cb6 (DEFAULT_TABLE table=test-table [id=dec43747cf7e4e6f9dd07f7eaca3c32b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:29:05.602975 22790 tablet_service.cc:1467] Processing CreateTablet for tablet a2cc56ba98d149429f05d6635e090cb6 (DEFAULT_TABLE table=test-table [id=dec43747cf7e4e6f9dd07f7eaca3c32b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:29:05.603382 22924 tablet_service.cc:1467] Processing CreateTablet for tablet a2cc56ba98d149429f05d6635e090cb6 (DEFAULT_TABLE table=test-table [id=dec43747cf7e4e6f9dd07f7eaca3c32b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:29:05.604251 22647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2cc56ba98d149429f05d6635e090cb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:29:05.604285 22790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2cc56ba98d149429f05d6635e090cb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:29:05.604612 22924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2cc56ba98d149429f05d6635e090cb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:29:05.616356 23008 tablet_bootstrap.cc:492] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3: Bootstrap starting.
I20260618 22:29:05.619516 23008 tablet_bootstrap.cc:654] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3: Neither blocks nor log segments found. Creating new log.
I20260618 22:29:05.620509 23008 log.cc:824] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3: Log is configured to *not* fsync() on all Append() calls
I20260618 22:29:05.622668 23010 tablet_bootstrap.cc:492] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77: Bootstrap starting.
I20260618 22:29:05.623107 23009 tablet_bootstrap.cc:492] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de: Bootstrap starting.
I20260618 22:29:05.623886 23008 tablet_bootstrap.cc:492] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3: No bootstrap required, opened a new log
I20260618 22:29:05.624163 23008 ts_tablet_manager.cc:1397] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260618 22:29:05.626042 23009 tablet_bootstrap.cc:654] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de: Neither blocks nor log segments found. Creating new log.
I20260618 22:29:05.626993 23010 tablet_bootstrap.cc:654] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77: Neither blocks nor log segments found. Creating new log.
I20260618 22:29:05.627151 23009 log.cc:824] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de: Log is configured to *not* fsync() on all Append() calls
I20260618 22:29:05.628176 23010 log.cc:824] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77: Log is configured to *not* fsync() on all Append() calls
I20260618 22:29:05.629837 23009 tablet_bootstrap.cc:492] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de: No bootstrap required, opened a new log
I20260618 22:29:05.630084 23009 ts_tablet_manager.cc:1397] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260618 22:29:05.631676 23010 tablet_bootstrap.cc:492] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77: No bootstrap required, opened a new log
I20260618 22:29:05.631942 23010 ts_tablet_manager.cc:1397] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260618 22:29:05.634014 23008 raft_consensus.cc:348] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a6b38cb1843c3918e7c61a779d8d3" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35637 } } peers { permanent_uuid: "fa730adf1dbb4bce9a9de421623a69de" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36829 } } peers { permanent_uuid: "6315f30a2f35495ba4707f62e51e6a77" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41693 } }
I20260618 22:29:05.634574 23008 raft_consensus.cc:374] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:29:05.634723 23008 raft_consensus.cc:729] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 941a6b38cb1843c3918e7c61a779d8d3, State: Initialized, Role: FOLLOWER
I20260618 22:29:05.635371 23008 consensus_queue.cc:260] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a6b38cb1843c3918e7c61a779d8d3" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35637 } } peers { permanent_uuid: "fa730adf1dbb4bce9a9de421623a69de" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36829 } } peers { permanent_uuid: "6315f30a2f35495ba4707f62e51e6a77" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41693 } }
I20260618 22:29:05.637005 23008 ts_tablet_manager.cc:1428] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260618 22:29:05.638537 23009 raft_consensus.cc:348] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a6b38cb1843c3918e7c61a779d8d3" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35637 } } peers { permanent_uuid: "fa730adf1dbb4bce9a9de421623a69de" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36829 } } peers { permanent_uuid: "6315f30a2f35495ba4707f62e51e6a77" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41693 } }
I20260618 22:29:05.638932 23009 raft_consensus.cc:374] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:29:05.639025 23009 raft_consensus.cc:729] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa730adf1dbb4bce9a9de421623a69de, State: Initialized, Role: FOLLOWER
I20260618 22:29:05.639621 23009 consensus_queue.cc:260] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a6b38cb1843c3918e7c61a779d8d3" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35637 } } peers { permanent_uuid: "fa730adf1dbb4bce9a9de421623a69de" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36829 } } peers { permanent_uuid: "6315f30a2f35495ba4707f62e51e6a77" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41693 } }
I20260618 22:29:05.640965 22855 heartbeater.cc:503] Master 127.10.242.254:45615 was elected leader, sending a full tablet report...
I20260618 22:29:05.641374 23009 ts_tablet_manager.cc:1428] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260618 22:29:05.642234 23010 raft_consensus.cc:348] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a6b38cb1843c3918e7c61a779d8d3" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35637 } } peers { permanent_uuid: "fa730adf1dbb4bce9a9de421623a69de" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36829 } } peers { permanent_uuid: "6315f30a2f35495ba4707f62e51e6a77" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41693 } }
I20260618 22:29:05.642650 23010 raft_consensus.cc:374] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:29:05.642755 23010 raft_consensus.cc:729] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6315f30a2f35495ba4707f62e51e6a77, State: Initialized, Role: FOLLOWER
I20260618 22:29:05.643349 23010 consensus_queue.cc:260] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a6b38cb1843c3918e7c61a779d8d3" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35637 } } peers { permanent_uuid: "fa730adf1dbb4bce9a9de421623a69de" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36829 } } peers { permanent_uuid: "6315f30a2f35495ba4707f62e51e6a77" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41693 } }
I20260618 22:29:05.644311 22989 heartbeater.cc:503] Master 127.10.242.254:45615 was elected leader, sending a full tablet report...
I20260618 22:29:05.644876 23010 ts_tablet_manager.cc:1428] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77: Time spent starting tablet: real 0.013s user 0.007s sys 0.006s
W20260618 22:29:05.655845 22713 tablet.cc:2369] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:29:05.668606 23016 raft_consensus.cc:482] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:29:05.668880 23016 raft_consensus.cc:504] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a6b38cb1843c3918e7c61a779d8d3" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35637 } } peers { permanent_uuid: "fa730adf1dbb4bce9a9de421623a69de" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36829 } } peers { permanent_uuid: "6315f30a2f35495ba4707f62e51e6a77" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41693 } }
I20260618 22:29:05.670885 23016 leader_election.cc:288] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 941a6b38cb1843c3918e7c61a779d8d3 (127.10.242.193:35637), fa730adf1dbb4bce9a9de421623a69de (127.10.242.194:36829)
I20260618 22:29:05.672870 23015 raft_consensus.cc:482] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:29:05.673168 23015 raft_consensus.cc:504] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a6b38cb1843c3918e7c61a779d8d3" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35637 } } peers { permanent_uuid: "fa730adf1dbb4bce9a9de421623a69de" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36829 } } peers { permanent_uuid: "6315f30a2f35495ba4707f62e51e6a77" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41693 } }
I20260618 22:29:05.675086 23015 leader_election.cc:288] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 941a6b38cb1843c3918e7c61a779d8d3 (127.10.242.193:35637), 6315f30a2f35495ba4707f62e51e6a77 (127.10.242.195:41693)
I20260618 22:29:05.677807 22992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.671092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:35637 (local address 127.10.242.195:53893)
0618 22:29:05.671329 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:05.671349 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:05.671486 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:05.671879 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:05.671884 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:05.671905 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:29:05.672066 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:05.672074 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.673399 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.673403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:05.675314 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:05.675330 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.677003 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.677009 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:05.677250 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.677286 (+ 36us) client_negotiation.cc:769] Sending connection context
0618 22:29:05.677393 (+ 107us) client_negotiation.cc:240] Negotiation successful
0618 22:29:05.677542 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":32}
I20260618 22:29:05.678054 23005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.671449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:53399 (local address 127.10.242.194:36829)
0618 22:29:05.671605 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:05.671611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:05.671844 (+ 233us) server_negotiation.cc:407] Connection header received
0618 22:29:05.671999 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:05.672003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:05.672058 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:05.672130 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:29:05.672594 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.673606 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.675458 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.676807 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.676933 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.677008 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:05.677627 (+ 619us) server_negotiation.cc:299] Negotiation successful
0618 22:29:05.677769 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260618 22:29:05.678431 23004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.671178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:53893 (local address 127.10.242.193:35637)
0618 22:29:05.671351 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:05.671357 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:05.671557 (+ 200us) server_negotiation.cc:407] Connection header received
0618 22:29:05.671654 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:05.671659 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:05.671751 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:05.671848 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:29:05.672228 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.673258 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.675492 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.676700 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.676844 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.676918 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:05.678052 (+ 1134us) server_negotiation.cc:299] Negotiation successful
0618 22:29:05.678199 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:29:05.678781 23017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.671170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:36829 (local address 127.10.242.195:53399)
0618 22:29:05.671767 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:05.671779 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:05.671879 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:05.672165 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:05.672169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:05.672194 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:29:05.672459 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:05.672466 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.673742 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.673745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:05.675292 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:05.675307 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.677224 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.677231 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:05.677423 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.677466 (+ 43us) client_negotiation.cc:769] Sending connection context
0618 22:29:05.678225 (+ 759us) client_negotiation.cc:240] Negotiation successful
0618 22:29:05.678386 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":7,"thread_start_us":435,"threads_started":1}
I20260618 22:29:05.679921 22810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2cc56ba98d149429f05d6635e090cb6" candidate_uuid: "6315f30a2f35495ba4707f62e51e6a77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa730adf1dbb4bce9a9de421623a69de" is_pre_election: true
I20260618 22:29:05.680763 22667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2cc56ba98d149429f05d6635e090cb6" candidate_uuid: "6315f30a2f35495ba4707f62e51e6a77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "941a6b38cb1843c3918e7c61a779d8d3" is_pre_election: true
I20260618 22:29:05.680891 22810 raft_consensus.cc:2454] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6315f30a2f35495ba4707f62e51e6a77 in term 0.
I20260618 22:29:05.681345 22667 raft_consensus.cc:2454] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6315f30a2f35495ba4707f62e51e6a77 in term 0.
I20260618 22:29:05.681734 22880 leader_election.cc:302] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6315f30a2f35495ba4707f62e51e6a77, fa730adf1dbb4bce9a9de421623a69de; no voters:
I20260618 22:29:05.682163 23016 raft_consensus.cc:2781] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:29:05.682333 23016 raft_consensus.cc:482] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:29:05.682426 23016 raft_consensus.cc:3037] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:29:05.685580 23016 raft_consensus.cc:504] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a6b38cb1843c3918e7c61a779d8d3" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35637 } } peers { permanent_uuid: "fa730adf1dbb4bce9a9de421623a69de" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36829 } } peers { permanent_uuid: "6315f30a2f35495ba4707f62e51e6a77" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41693 } }
I20260618 22:29:05.686291 23016 leader_election.cc:288] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [CANDIDATE]: Term 1 election: Requested vote from peers 941a6b38cb1843c3918e7c61a779d8d3 (127.10.242.193:35637), fa730adf1dbb4bce9a9de421623a69de (127.10.242.194:36829)
I20260618 22:29:05.686746 22667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2cc56ba98d149429f05d6635e090cb6" candidate_uuid: "6315f30a2f35495ba4707f62e51e6a77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "941a6b38cb1843c3918e7c61a779d8d3"
I20260618 22:29:05.687016 22810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2cc56ba98d149429f05d6635e090cb6" candidate_uuid: "6315f30a2f35495ba4707f62e51e6a77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa730adf1dbb4bce9a9de421623a69de"
I20260618 22:29:05.687261 22810 raft_consensus.cc:3037] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:29:05.687461 22667 raft_consensus.cc:3037] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:29:05.687558 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.677623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:46963 (local address 127.10.242.195:41693)
0618 22:29:05.677773 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:05.677778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:05.680901 (+ 3123us) server_negotiation.cc:407] Connection header received
0618 22:29:05.681201 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:05.681206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:05.681273 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:05.681334 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:29:05.682398 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.683350 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.685398 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.686563 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.686722 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.686794 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:05.687241 (+ 447us) server_negotiation.cc:299] Negotiation successful
0618 22:29:05.687350 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260618 22:29:05.688338 23018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.675824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:41693 (local address 127.10.242.194:46963)
0618 22:29:05.680810 (+ 4986us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:05.680828 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:05.680999 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:05.681506 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:05.681510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:05.681537 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:29:05.682273 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:05.682286 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.683530 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.683534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:05.685263 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:05.685275 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.686931 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.686938 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:05.687077 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.687115 (+ 38us) client_negotiation.cc:769] Sending connection context
0618 22:29:05.687743 (+ 628us) client_negotiation.cc:240] Negotiation successful
0618 22:29:05.687891 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4807,"mutex_wait_us":83,"thread_start_us":3397,"threads_started":1}
I20260618 22:29:05.688750 23004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.676198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:58387 (local address 127.10.242.193:35637)
0618 22:29:05.678635 (+ 2437us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:05.678640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:05.680807 (+ 2167us) server_negotiation.cc:407] Connection header received
0618 22:29:05.681102 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:05.681108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:05.681175 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:05.681233 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:29:05.682119 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.683129 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.685821 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.687558 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.687714 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.687787 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:05.688323 (+ 536us) server_negotiation.cc:299] Negotiation successful
0618 22:29:05.688461 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2339,"thread_start_us":3682,"threads_started":1}
I20260618 22:29:05.689440 22944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2cc56ba98d149429f05d6635e090cb6" candidate_uuid: "fa730adf1dbb4bce9a9de421623a69de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6315f30a2f35495ba4707f62e51e6a77" is_pre_election: true
I20260618 22:29:05.689843 22944 raft_consensus.cc:2379] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa730adf1dbb4bce9a9de421623a69de in current term 1: Already voted for candidate 6315f30a2f35495ba4707f62e51e6a77 in this term.
I20260618 22:29:05.690490 22667 raft_consensus.cc:2454] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6315f30a2f35495ba4707f62e51e6a77 in term 1.
I20260618 22:29:05.690611 23019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.675265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:35637 (local address 127.10.242.194:58387)
0618 22:29:05.680709 (+ 5444us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:05.680729 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:05.680909 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:05.681443 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:05.681447 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:05.681472 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:29:05.681923 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:05.681936 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.683464 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.683470 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:05.685669 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:05.685683 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.687929 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.687935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:05.688127 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.688159 (+ 32us) client_negotiation.cc:769] Sending connection context
0618 22:29:05.688956 (+ 797us) client_negotiation.cc:240] Negotiation successful
0618 22:29:05.689141 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5285,"mutex_wait_us":1,"spinlock_wait_cycles":29056,"thread_start_us":4418,"threads_started":1}
I20260618 22:29:05.690944 22877 leader_election.cc:302] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6315f30a2f35495ba4707f62e51e6a77, 941a6b38cb1843c3918e7c61a779d8d3; no voters:
I20260618 22:29:05.691254 22810 raft_consensus.cc:2454] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6315f30a2f35495ba4707f62e51e6a77 in term 1.
I20260618 22:29:05.691269 23016 raft_consensus.cc:2781] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:29:05.691340 22666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2cc56ba98d149429f05d6635e090cb6" candidate_uuid: "fa730adf1dbb4bce9a9de421623a69de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "941a6b38cb1843c3918e7c61a779d8d3" is_pre_election: true
I20260618 22:29:05.691721 22666 raft_consensus.cc:2379] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa730adf1dbb4bce9a9de421623a69de in current term 1: Already voted for candidate 6315f30a2f35495ba4707f62e51e6a77 in this term.
I20260618 22:29:05.692423 23016 raft_consensus.cc:686] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [term 1 LEADER]: Becoming Leader. State: Replica: 6315f30a2f35495ba4707f62e51e6a77, State: Running, Role: LEADER
I20260618 22:29:05.692551 22736 leader_election.cc:302] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa730adf1dbb4bce9a9de421623a69de; no voters: 6315f30a2f35495ba4707f62e51e6a77, 941a6b38cb1843c3918e7c61a779d8d3
I20260618 22:29:05.692955 23015 raft_consensus.cc:2726] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 22:29:05.692950 23016 consensus_queue.cc:237] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a6b38cb1843c3918e7c61a779d8d3" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35637 } } peers { permanent_uuid: "fa730adf1dbb4bce9a9de421623a69de" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36829 } } peers { permanent_uuid: "6315f30a2f35495ba4707f62e51e6a77" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41693 } }
I20260618 22:29:05.697875 22498 catalog_manager.cc:5495] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6315f30a2f35495ba4707f62e51e6a77 (127.10.242.195). New cstate: current_term: 1 leader_uuid: "6315f30a2f35495ba4707f62e51e6a77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a6b38cb1843c3918e7c61a779d8d3" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa730adf1dbb4bce9a9de421623a69de" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6315f30a2f35495ba4707f62e51e6a77" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41693 } health_report { overall_health: HEALTHY } } }
W20260618 22:29:05.703837 22856 tablet.cc:2369] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:29:05.720281 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.714936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49654 (local address 127.10.242.195:41693)
0618 22:29:05.715096 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:05.715102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:05.715192 (+ 90us) server_negotiation.cc:407] Connection header received
0618 22:29:05.715384 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:05.715388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:05.715461 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:05.715587 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:29:05.715972 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.716734 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.718170 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.718568 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.718674 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.719214 (+ 540us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:29:05.719702 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:29:05.719839 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:05.719924 (+ 85us) server_negotiation.cc:299] Negotiation successful
0618 22:29:05.720000 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:29:05.728223 22666 raft_consensus.cc:1261] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3 [term 1 FOLLOWER]: Refusing update from remote peer 6315f30a2f35495ba4707f62e51e6a77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:29:05.728268 22810 raft_consensus.cc:1261] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de [term 1 FOLLOWER]: Refusing update from remote peer 6315f30a2f35495ba4707f62e51e6a77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:29:05.729038 23021 consensus_queue.cc:1035] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa730adf1dbb4bce9a9de421623a69de" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:29:05.729377 23016 consensus_queue.cc:1035] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "941a6b38cb1843c3918e7c61a779d8d3" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:29:05.755297 22498 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:29:05.755513 22498 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:29:05.759256 22498 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa1d00a790eb408d92751b416c131b16: Sending DeleteTablet for 3 replicas of tablet a2cc56ba98d149429f05d6635e090cb6
I20260618 22:29:05.760454 22790 tablet_service.cc:1514] Processing DeleteTablet for tablet a2cc56ba98d149429f05d6635e090cb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:29:05 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:36386
I20260618 22:29:05.760663 22647 tablet_service.cc:1514] Processing DeleteTablet for tablet a2cc56ba98d149429f05d6635e090cb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:29:05 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:50186
I20260618 22:29:05.760844 22924 tablet_service.cc:1514] Processing DeleteTablet for tablet a2cc56ba98d149429f05d6635e090cb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:29:05 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:49638
I20260618 22:29:05.764992 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 22579
I20260618 22:29:05.773510 23034 tablet_replica.cc:330] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de: stopping tablet replica
I20260618 22:29:05.773633 23036 tablet_replica.cc:330] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77: stopping tablet replica
I20260618 22:29:05.774030 23034 raft_consensus.cc:2229] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:29:05.774266 23036 raft_consensus.cc:2229] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:29:05.774801 23035 tablet_replica.cc:330] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3: stopping tablet replica
I20260618 22:29:05.775066 23036 raft_consensus.cc:2258] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:29:05.775277 23035 raft_consensus.cc:2229] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:29:05.775858 23035 raft_consensus.cc:2258] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:29:05.775861 23034 raft_consensus.cc:2258] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:29:05.778028 23035 ts_tablet_manager.cc:1905] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:29:05.778026 23036 ts_tablet_manager.cc:1905] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:29:05.783085 23034 ts_tablet_manager.cc:1905] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:29:05.786492 23035 ts_tablet_manager.cc:1918] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:29:05.786759 23035 log.cc:1196] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/a2cc56ba98d149429f05d6635e090cb6
I20260618 22:29:05.787487 23035 ts_tablet_manager.cc:1939] T a2cc56ba98d149429f05d6635e090cb6 P 941a6b38cb1843c3918e7c61a779d8d3: Deleting consensus metadata
I20260618 22:29:05.789511 22483 catalog_manager.cc:4841] TS 941a6b38cb1843c3918e7c61a779d8d3 (127.10.242.193:35637): tablet a2cc56ba98d149429f05d6635e090cb6 (table test-table [id=dec43747cf7e4e6f9dd07f7eaca3c32b]) successfully deleted
Jun 18 22:29:05 dist-test-slave-0956 krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:29:05.792080 23034 ts_tablet_manager.cc:1918] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:29:05.792225 23034 log.cc:1196] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/a2cc56ba98d149429f05d6635e090cb6
I20260618 22:29:05.792630 23034 ts_tablet_manager.cc:1939] T a2cc56ba98d149429f05d6635e090cb6 P fa730adf1dbb4bce9a9de421623a69de: Deleting consensus metadata
I20260618 22:29:05.794245 22486 catalog_manager.cc:4841] TS fa730adf1dbb4bce9a9de421623a69de (127.10.242.194:36829): tablet a2cc56ba98d149429f05d6635e090cb6 (table test-table [id=dec43747cf7e4e6f9dd07f7eaca3c32b]) successfully deleted
I20260618 22:29:05.794266 23036 ts_tablet_manager.cc:1918] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:29:05.794464 23036 log.cc:1196] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/a2cc56ba98d149429f05d6635e090cb6
I20260618 22:29:05.795066 23036 ts_tablet_manager.cc:1939] T a2cc56ba98d149429f05d6635e090cb6 P 6315f30a2f35495ba4707f62e51e6a77: Deleting consensus metadata
I20260618 22:29:05.797003 22485 catalog_manager.cc:4841] TS 6315f30a2f35495ba4707f62e51e6a77 (127.10.242.195:41693): tablet a2cc56ba98d149429f05d6635e090cb6 (table test-table [id=dec43747cf7e4e6f9dd07f7eaca3c32b]) successfully deleted
I20260618 22:29:05.801021 23004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.766520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33292 (local address 127.10.242.193:35637)
0618 22:29:05.766700 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:05.766706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:05.781213 (+ 14507us) server_negotiation.cc:407] Connection header received
0618 22:29:05.781443 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:05.781447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:05.781544 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:05.781631 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:05.783272 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.784374 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.786404 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.787010 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.787114 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.795127 (+ 8013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:05.795146 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:05.795166 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:05.795211 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:05.798056 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:05.798836 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:05.798844 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:05.798847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:05.798977 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:05.799397 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:05.799401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:05.799403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:05.799772 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:05.800076 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:05.800495 (+ 419us) server_negotiation.cc:299] Negotiation successful
0618 22:29:05.800714 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:29:05.801780 22707 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:29:05.891974 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 22579
I20260618 22:29:05.906178 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 22718
Jun 18 22:29:05 dist-test-slave-0956 krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.194@KRBTEST.COM
I20260618 22:29:05.920933 23005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:05.906930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42298 (local address 127.10.242.194:36829)
0618 22:29:05.907198 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:05.907203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:05.907221 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:29:05.907359 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:05.907363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:05.907425 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:05.907519 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:05.908758 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.909552 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.910848 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:05.911244 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:05.911353 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:05.915491 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:05.915506 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:05.915521 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:05.915554 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:05.918294 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:05.919014 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:05.919022 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:05.919024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:05.919143 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:05.919486 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:05.919491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:05.919494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:05.919838 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:05.920120 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:05.920388 (+ 268us) server_negotiation.cc:299] Negotiation successful
0618 22:29:05.920601 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260618 22:29:05.921624 22850 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:29:06.015822 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 22718
I20260618 22:29:06.028431 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 22861
Jun 18 22:29:06 dist-test-slave-0956 krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.195@KRBTEST.COM
I20260618 22:29:06.044473 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:06.029231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60548 (local address 127.10.242.195:41693)
0618 22:29:06.029473 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:06.029479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:06.029497 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:29:06.029600 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:06.029605 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:06.029676 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:06.029760 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:06.031032 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:06.031783 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:06.033004 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:06.033535 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:06.033651 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:06.038517 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:06.038538 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:06.038554 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:06.038601 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:06.041673 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:06.042455 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:06.042462 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:06.042465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:06.042596 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:06.042996 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:06.043000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:06.043002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:06.043376 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:06.043693 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:06.043964 (+ 271us) server_negotiation.cc:299] Negotiation successful
0618 22:29:06.044171 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260618 22:29:06.045367 22984 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:29:06.164839 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 22861
I20260618 22:29:06.179747 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 22467
I20260618 22:29:06.180907 22529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:29:06.289196 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 22467
I20260618 22:29:06.307668 11211 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260618 22:29:06.642850 11211 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260618 22:29:06.643055 11211 mini_ranger.cc:66] Stopping Ranger...
I20260618 22:29:06.978407 11211 mini_ranger.cc:68] Stopped Ranger
2026-06-18 22:29:06.978 UTC [22053] LOG: received smart shutdown request
2026-06-18 22:29:06.981 UTC [22053] LOG: background worker "logical replication launcher" (PID 22062) exited with exit code 1
2026-06-18 22:29:06.982 UTC [22057] LOG: shutting down
2026-06-18 22:29:07.147 UTC [22053] LOG: database system is shut down
2026-06-18T22:29:07Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48894 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:29:07 dist-test-slave-0956 krb5kdc[23050](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:29:07 dist-test-slave-0956 krb5kdc[23050](info): set up 2 sockets
Jun 18 22:29:07 dist-test-slave-0956 krb5kdc[23050](info): commencing operation
krb5kdc: starting...
W20260618 22:29:08.273126 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:29:08 dist-test-slave-0956 krb5kdc[23050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:29:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:29:08Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.242.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.242.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.242.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/rangeradmin_127.10.242.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.242.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/rangeradmin_127.10.242.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.242.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.242.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.242.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/rangerlookup_127.10.242.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.242.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/rangerlookup_127.10.242.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.242.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.242.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.242.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/HTTP_127.10.242.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.242.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/HTTP_127.10.242.212@KRBTEST.COM.keytab.
I20260618 22:29:08.580044 11211 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJpXzaW/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/postgres -l logfile start
2026-06-18 22:29:11.074 UTC [23080] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-18 22:29:11.074 UTC [23080] LOG: listening on IPv4 address "127.10.242.212", port 52429
2026-06-18 22:29:11.078 UTC [23080] LOG: listening on Unix socket "/tmp/.s.PGSQL.52429"
2026-06-18 22:29:11.101 UTC [23083] LOG: database system was shut down at 2026-06-18 22:29:09 UTC
2026-06-18 22:29:11.105 UTC [23080] LOG: database system is ready to accept connections
I20260618 22:29:11.992957 11211 mini_postgres.cc:96] Postgres bound to 52429
2026-06-18 22:29:12.010 UTC [23091] FATAL: database "slave" does not exist
127.10.242.212:52429 - accepting connections
I20260618 22:29:12.011008 11211 mini_ranger.cc:161] Starting Ranger...
I20260618 22:29:12.035717 11211 mini_ranger.cc:84] Created miniranger Postgres user
I20260618 22:29:12.502959 11211 mini_ranger.cc:87] Created ranger Postgres database
I20260618 22:29:12.503068 11211 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-admin
2026-06-18 22:29:12,790 [I] DB FLAVOR :POSTGRES
2026-06-18 22:29:12,791 [I] --------- Verifying Ranger DB connection ---------
2026-06-18 22:29:12,791 [I] Checking connection..
2026-06-18 22:29:12,791 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-18 22:29:13,054 [I] Checking connection passed.
2026-06-18 22:29:13,054 [I] --------- Verifying version history table ---------
2026-06-18 22:29:13,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-18 22:29:13,314 [I] Table x_db_version_h does not exist in database ranger
2026-06-18 22:29:13,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-18 22:29:13,553 [I] Table x_db_version_h does not exist in database ranger
2026-06-18 22:29:13,553 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-18 22:29:13,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-18 22:29:13.788 UTC [23155] WARNING: there is no transaction in progress
2026-06-18 22:29:13,795 [I] create_dbversion_catalog.sql file imported successfully
2026-06-18 22:29:13,795 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-18 22:29:14,035 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-18 22:29:14,035 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-18 22:29:14,036 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-18 22:29:14,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-18 22:29:14,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-0956.c.gcp-upstream.internal','N') ;"
2026-06-18 22:29:14,772 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-18 22:29:15,015 [I] Table x_portal_user does not exist in database ranger
2026-06-18 22:29:15,015 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-18 22:29:15,276 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-18 22:29:15,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-18 22:29:15,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-18 22:29:15,792 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-18 22:29:15,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-18 22:29:16.788 UTC [23263] WARNING: there is no transaction in progress
2026-06-18 22:29:16.805 UTC [23263] WARNING: there is no transaction in progress
2026-06-18 22:29:16.822 UTC [23263] WARNING: there is no transaction in progress
2026-06-18 22:29:16.839 UTC [23263] WARNING: there is no transaction in progress
2026-06-18 22:29:16.857 UTC [23263] WARNING: there is no transaction in progress
2026-06-18 22:29:16.876 UTC [23263] WARNING: there is no transaction in progress
2026-06-18 22:29:16.932 UTC [23263] WARNING: there is no transaction in progress
2026-06-18 22:29:16.940 UTC [23263] WARNING: there is no transaction in progress
2026-06-18 22:29:16.955 UTC [23263] WARNING: there is no transaction in progress
2026-06-18 22:29:16.966 UTC [23263] WARNING: there is no transaction in progress
2026-06-18 22:29:16.976 UTC [23263] WARNING: there is no transaction in progress
2026-06-18 22:29:16.991 UTC [23263] WARNING: there is no transaction in progress
2026-06-18 22:29:17.001 UTC [23263] WARNING: there is no transaction in progress
2026-06-18 22:29:17.008 UTC [23263] WARNING: there is no transaction in progress
2026-06-18 22:29:17.015 UTC [23263] WARNING: there is no transaction in progress
2026-06-18 22:29:17.022 UTC [23263] WARNING: there is no transaction in progress
2026-06-18 22:29:17,650 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-18 22:29:17,650 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-18 22:29:17,874 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-18 22:29:17,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-18 22:29:18,130 [I] Table x_portal_user already exists in database 'ranger'
2026-06-18 22:29:18,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-18 22:29:18,397 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-18 22:29:18,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-18 22:29:18,656 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-18 22:29:18,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-0956.c.gcp-upstream.internal';"
2026-06-18 22:29:19,133 [I] CORE_DB_SCHEMA import status has been updated
2026-06-18 22:29:19,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-18 22:29:19,389 [I] DB_PATCHES have already been applied
I20260618 22:29:19.395313 11211 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJpXzaW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260618 22:29:19.395398 11211 mini_ranger.cc:193] Using host: 127.10.242.212
I20260618 22:29:19.408739 11211 mini_ranger.cc:227] Ranger admin URL: http://127.10.242.212:58005
Jun 18, 2026 10:29:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 18, 2026 10:29:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 18, 2026 10:29:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-admin/ews/webapp
Jun 18, 2026 10:29:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 18, 2026 10:29:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 18, 2026 10:29:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-admin/ews/webapp].
Jun 18, 2026 10:29:20 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260618 22:29:20.470518 11211 mini_ranger.cc:160] Time spent starting Ranger: real 8.460s user 0.000s sys 0.038s
Jun 18, 2026 10:29:40 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 18, 2026 10:29:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Jun 18, 2026 10:29:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 18, 2026 10:29:40 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 18, 2026 10:29:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 18, 2026 10:29:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 18, 2026 10:29:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 18, 2026 10:29:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 18, 2026 10:29:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 18, 2026 10:29:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 18, 2026 10:29:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 18, 2026 10:29:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 18, 2026 10:29:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 18, 2026 10:29:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 18, 2026 10:29:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 18, 2026 10:29:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 18, 2026 10:29:40 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 18, 2026 10:29:41 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260618 22:29:42.408695 11211 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.242.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.242.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.242.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/rangerkms_127.10.242.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.242.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/rangerkms_127.10.242.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.242.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.242.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.242.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/HTTP_127.10.242.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.242.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/HTTP_127.10.242.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 18 22:29:42 dist-test-slave-0956 krb5kdc[23050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821782, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260618 22:29:42.580848 11211 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260618 22:29:42.603169 11211 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260618 22:29:43.473393 11211 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260618 22:29:43.473500 11211 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-kms
I20260618 22:29:43.473541 11211 mini_ranger_kms.cc:227] Using postgres at 127.10.242.212:52429
2026-06-18 22:29:43,811 [I] DB FLAVOR :POSTGRES
2026-06-18 22:29:43,811 [I] --------- Verifying Ranger DB connection ---------
2026-06-18 22:29:43,811 [I] Checking connection
2026-06-18 22:29:43,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-18 22:29:44,044 [I] connection success
2026-06-18 22:29:44,044 [I] --------- Verifying Ranger DB tables ---------
2026-06-18 22:29:44,044 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-18 22:29:44,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-18 22:29:44,280 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-18 22:29:44,280 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-18 22:29:44,280 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-18 22:29:44,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.212:52429/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-18 22:29:44,544 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260618 22:29:45.011425 11211 mini_ranger_kms.cc:310] Created kms service
I20260618 22:29:45.191977 11211 mini_ranger_kms.cc:326] Created kudu user
I20260618 22:29:45.234035 11211 mini_ranger_kms.cc:343] Created rangerkms user
Jun 18 22:29:45 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821782, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
I20260618 22:29:45.647868 11211 mini_ranger_kms.cc:384] Added ranger policy
I20260618 22:29:45.648190 11211 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJpXzaW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJpXzaW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260618 22:29:45.648234 11211 mini_ranger_kms.cc:240] Using host: 127.10.242.212
I20260618 22:29:45.662096 11211 mini_ranger_kms.cc:276] Ranger KMS PID: 23443
I20260618 22:29:45.662235 11211 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.242.212:34423
Jun 18, 2026 10:29:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 18, 2026 10:29:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 18, 2026 10:29:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 18, 2026 10:29:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 18, 2026 10:29:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-kms/ews/webapp].
Jun 18, 2026 10:29:47 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260618 22:29:47.764649 11211 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.184s user 0.063s sys 0.293s
I20260618 22:29:47.764855 11211 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260618 22:29:47.765007 11211 mini_ranger_kms.cc:401] 127.10.242.212:34423/kms/v1/keys
Jun 18 22:29:47 dist-test-slave-0956 krb5kdc[23050](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781821787, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.242.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 18 22:29:48 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821787, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.242.212@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
Jun 18 22:29:48 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821787, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.242.212@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
Jun 18 22:29:50 dist-test-slave-0956 krb5kdc[23050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:29:51.005255 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:41203
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:41203
--ranger_config_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.212:34423/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:29:51.329473 23494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:29:51.329783 23494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:29:51.329906 23494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:29:51.337265 23494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:29:51.337389 23494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:29:51.337459 23494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:29:51.337532 23494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:29:51.337602 23494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:29:51.347684 23494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.242.212:34423/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:41203
--ranger_config_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:41203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:29:51.348963 23494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:29:51.350271 23494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:29:51.361308 23499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:51.361277 23502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:51.361281 23500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:29:51.362094 23494 server_base.cc:1034] running on GCE node
I20260618 22:29:51.362843 23494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:29:51.364205 23494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:29:51.365393 23494 hybrid_clock.cc:648] HybridClock initialized: now 1781821791365343 us; error 49 us; skew 500 ppm
I20260618 22:29:51.365761 23494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:29:51 dist-test-slave-0956 krb5kdc[23050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821791, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:29:51.368687 23494 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:29:51.370033 23494 webserver.cc:460] Webserver started at http://127.10.242.254:34123/ using document root <none> and password file <none>
I20260618 22:29:51.370640 23494 fs_manager.cc:359] Metadata directory not provided
I20260618 22:29:51.370751 23494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:29:51.371018 23494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:29:51 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821791, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
I20260618 22:29:51.427201 23494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "fabb7980629643faaa413a42d1b5d1f2"
format_stamp: "Formatted at 2026-06-18 22:29:51 on dist-test-slave-0956"
server_key: "f309f646f7d1bcf1c874b35340376f4f"
server_key_iv: "3544a8c2e72528bde08e0fa16e1528fb"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:51.428140 23494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "fabb7980629643faaa413a42d1b5d1f2"
format_stamp: "Formatted at 2026-06-18 22:29:51 on dist-test-slave-0956"
server_key: "f309f646f7d1bcf1c874b35340376f4f"
server_key_iv: "3544a8c2e72528bde08e0fa16e1528fb"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:51.433888 23494 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260618 22:29:51.454459 23509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:51.455955 23494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:29:51.456128 23494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "fabb7980629643faaa413a42d1b5d1f2"
format_stamp: "Formatted at 2026-06-18 22:29:51 on dist-test-slave-0956"
server_key: "f309f646f7d1bcf1c874b35340376f4f"
server_key_iv: "3544a8c2e72528bde08e0fa16e1528fb"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:51.456341 23494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:29:51.490969 23494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:29:51.494946 23494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:29:51.495230 23494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:29:51.512753 23494 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:41203
I20260618 22:29:51.512773 23560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:41203 every 8 connection(s)
I20260618 22:29:51.514108 23494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:29:51.517925 23561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:29:51.520351 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 23494
I20260618 22:29:51.520551 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:29:51.521459 11211 external_mini_cluster.cc:1428] Setting key d923dc6cddfb96dbe25e99796a1d4565
I20260618 22:29:51.530232 23561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2: Bootstrap starting.
I20260618 22:29:51.533990 23561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2: Neither blocks nor log segments found. Creating new log.
I20260618 22:29:51.535274 23561 log.cc:824] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:29:51 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:29:51.538367 23561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2: No bootstrap required, opened a new log
I20260618 22:29:51.544056 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:51.523630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55140 (local address 127.10.242.254:41203)
0618 22:29:51.524380 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:51.524393 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:51.524434 (+ 41us) server_negotiation.cc:407] Connection header received
0618 22:29:51.526107 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:51.526130 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:51.526525 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:51.526866 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:51.528404 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:51.529718 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:51.531495 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:51.532181 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:51.532315 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:51.536800 (+ 4485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:51.536808 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:51.536823 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:51.536858 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:51.539632 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:51.540280 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:51.540286 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:51.540292 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:51.540450 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:51.540828 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:51.540833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:51.540836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:51.541428 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:51.541721 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:51.542164 (+ 443us) server_negotiation.cc:299] Negotiation successful
0618 22:29:51.542618 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":345,"threads_started":1}
I20260618 22:29:51.547577 23561 raft_consensus.cc:348] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabb7980629643faaa413a42d1b5d1f2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 41203 } }
I20260618 22:29:51.547914 23561 raft_consensus.cc:374] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:29:51.547988 23561 raft_consensus.cc:729] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb7980629643faaa413a42d1b5d1f2, State: Initialized, Role: FOLLOWER
I20260618 22:29:51.548477 23561 consensus_queue.cc:260] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabb7980629643faaa413a42d1b5d1f2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 41203 } }
I20260618 22:29:51.548676 23561 raft_consensus.cc:388] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:29:51.548801 23561 raft_consensus.cc:482] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:29:51.548952 23561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:29:51.551282 23561 raft_consensus.cc:504] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabb7980629643faaa413a42d1b5d1f2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 41203 } }
I20260618 22:29:51.551753 23561 leader_election.cc:302] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fabb7980629643faaa413a42d1b5d1f2; no voters:
I20260618 22:29:51.552290 23561 leader_election.cc:288] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:29:51.552423 23566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:29:51.553596 23566 raft_consensus.cc:686] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [term 1 LEADER]: Becoming Leader. State: Replica: fabb7980629643faaa413a42d1b5d1f2, State: Running, Role: LEADER
I20260618 22:29:51.554158 23566 consensus_queue.cc:237] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabb7980629643faaa413a42d1b5d1f2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 41203 } }
I20260618 22:29:51.554438 23561 sys_catalog.cc:564] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:29:51.556792 23568 sys_catalog.cc:455] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fabb7980629643faaa413a42d1b5d1f2. Latest consensus state: current_term: 1 leader_uuid: "fabb7980629643faaa413a42d1b5d1f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabb7980629643faaa413a42d1b5d1f2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 41203 } } }
I20260618 22:29:51.557205 23568 sys_catalog.cc:458] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [sys.catalog]: This master's current role is: LEADER
I20260618 22:29:51.557660 23567 sys_catalog.cc:455] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fabb7980629643faaa413a42d1b5d1f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabb7980629643faaa413a42d1b5d1f2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 41203 } } }
I20260618 22:29:51.558094 23567 sys_catalog.cc:458] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [sys.catalog]: This master's current role is: LEADER
I20260618 22:29:51.559500 23574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:29:51.564376 23574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:29:51.566949 23561 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260618 22:29:51.584005 23574 catalog_manager.cc:1347] Generated new cluster ID: db430e83521d46b5814e3442171a21ae
I20260618 22:29:51.584146 23574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:29:51.616556 23574 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:29:51.618068 23574 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:29:51.630235 23574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2: Generated new TSK 0
I20260618 22:29:51.631477 23574 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
I20260618 22:29:52.372143 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.212:34423/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:41203
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:29:52 dist-test-slave-0956 krb5kdc[23050](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781821792, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260618 22:29:52.818461 23606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:29:52.818964 23606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:29:52.819191 23606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:29:52.833616 23606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:29:52.833768 23606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:29:52.833941 23606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:29:52.844080 23606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.242.212:34423/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:41203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:29:52.845781 23606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:29:52.847544 23606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:29:52.872032 23616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:52.873760 23617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:29:52.882402 23606 server_base.cc:1034] running on GCE node
W20260618 22:29:52.882833 23619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:29:52.883785 23606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:29:52.888234 23606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:29:52.889560 23606 hybrid_clock.cc:648] HybridClock initialized: now 1781821792889454 us; error 106 us; skew 500 ppm
I20260618 22:29:52.889989 23606 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:29:52 dist-test-slave-0956 krb5kdc[23050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821792, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:29:52.894174 23606 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:29:52.896252 23606 webserver.cc:460] Webserver started at http://127.10.242.193:34777/ using document root <none> and password file <none>
I20260618 22:29:52.897233 23606 fs_manager.cc:359] Metadata directory not provided
I20260618 22:29:52.897444 23606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:29:52.897845 23606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:29:52 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821792, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
I20260618 22:29:52.917984 23606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "ae65410218aa462ca85d19dc8dc779e9"
format_stamp: "Formatted at 2026-06-18 22:29:52 on dist-test-slave-0956"
server_key: "7104310e9977ac2c39fcc6744896ffec"
server_key_iv: "4aef98afd06321e8f41f0edfc332a593"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:52.919780 23606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "ae65410218aa462ca85d19dc8dc779e9"
format_stamp: "Formatted at 2026-06-18 22:29:52 on dist-test-slave-0956"
server_key: "7104310e9977ac2c39fcc6744896ffec"
server_key_iv: "4aef98afd06321e8f41f0edfc332a593"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:52.930837 23606 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.003s sys 0.007s
I20260618 22:29:52.959892 23626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:52.963291 23606 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.009s sys 0.001s
I20260618 22:29:52.963523 23606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "ae65410218aa462ca85d19dc8dc779e9"
format_stamp: "Formatted at 2026-06-18 22:29:52 on dist-test-slave-0956"
server_key: "7104310e9977ac2c39fcc6744896ffec"
server_key_iv: "4aef98afd06321e8f41f0edfc332a593"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:52.963819 23606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:29:53.015106 23606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:29:53.020666 23606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:29:53.021065 23606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:29:53.028437 23606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:29:53.037521 23606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:29:53.037705 23606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260618 22:29:53.037909 23606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:29:53.038023 23606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:29:53 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821792, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
I20260618 22:29:53.156031 23606 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:41063
I20260618 22:29:53.156057 23738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:41063 every 8 connection(s)
I20260618 22:29:53.157799 23606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:29:53.164126 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 23606
I20260618 22:29:53.164315 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:29:53.165345 11211 external_mini_cluster.cc:1428] Setting key 5b2e1b24b35d860613d6ec5e62bcd5c6
Jun 18 22:29:53 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821792, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:29:53.210214 23741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:53.161410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:55111 (local address 127.10.242.254:41203)
0618 22:29:53.190420 (+ 29010us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:53.190427 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:53.190456 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:29:53.191077 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:53.191084 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:53.191203 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:53.191300 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:53.194245 (+ 2945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:53.195408 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:53.197716 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:53.198348 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:53.198471 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:53.202899 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:53.202907 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:53.202911 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:53.202967 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:53.206501 (+ 3534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:53.207496 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:53.207502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:53.207505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:53.207656 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:53.208142 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:53.208147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:53.208149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:53.208498 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:53.208633 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:53.209590 (+ 957us) server_negotiation.cc:299] Negotiation successful
0618 22:29:53.209811 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28866,"thread_start_us":29036,"threads_started":1}
I20260618 22:29:53.212570 23742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:53.162108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:41203 (local address 127.10.242.193:55111)
0618 22:29:53.186392 (+ 24284us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:53.186458 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:53.190581 (+ 4123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:53.191799 (+ 1218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:53.191812 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:53.192472 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:29:53.193353 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:53.193398 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:53.195572 (+ 2174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:53.195578 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:53.197553 (+ 1975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:53.197579 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:53.198597 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:53.198604 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:53.198878 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:53.199674 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:29:53.199704 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:29:53.202614 (+ 2910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:29:53.206689 (+ 4075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:29:53.206698 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:29:53.206719 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:29:53.207356 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:29:53.207782 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:29:53.207786 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:29:53.207789 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:29:53.208027 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:29:53.208697 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:29:53.208705 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:29:53.209171 (+ 466us) client_negotiation.cc:769] Sending connection context
0618 22:29:53.210544 (+ 1373us) client_negotiation.cc:240] Negotiation successful
0618 22:29:53.211108 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23891,"thread_start_us":26887,"threads_started":1}
I20260618 22:29:53.214663 23739 heartbeater.cc:347] Connected to a master server at 127.10.242.254:41203
I20260618 22:29:53.214994 23739 heartbeater.cc:464] Registering TS with master...
I20260618 22:29:53.215895 23739 heartbeater.cc:511] Master 127.10.242.254:41203 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.242.194@KRBTEST.COM; defaulting to no policy
I20260618 22:29:53.218250 23526 ts_manager.cc:194] Registered new tserver with Master: ae65410218aa462ca85d19dc8dc779e9 (127.10.242.193:41063)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.194@KRBTEST.COM" created.
I20260618 22:29:53.220060 23526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:55111
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
I20260618 22:29:53.250533 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.212:34423/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:41203
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:29:53 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821792, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
W20260618 22:29:53.750967 23745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:29:53.751478 23745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:29:53.752056 23745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:29:53.763947 23745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:29:53.764287 23745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:29:53.764533 23745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:29:53.780431 23745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.242.212:34423/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:41203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:29:53.782511 23745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:29:53.784248 23745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:29:53.800664 23759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:53.800664 23760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:53.807921 23762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:29:53.808833 23745 server_base.cc:1034] running on GCE node
I20260618 22:29:53.809648 23745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:29:53.811003 23745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:29:53.812256 23745 hybrid_clock.cc:648] HybridClock initialized: now 1781821793812202 us; error 77 us; skew 500 ppm
I20260618 22:29:53.812641 23745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:29:53 dist-test-slave-0956 krb5kdc[23050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821793, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:29:53.816821 23745 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:29:53.818761 23745 webserver.cc:460] Webserver started at http://127.10.242.194:42083/ using document root <none> and password file <none>
I20260618 22:29:53.819618 23745 fs_manager.cc:359] Metadata directory not provided
I20260618 22:29:53.819752 23745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:29:53.820073 23745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:29:53 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821793, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
I20260618 22:29:53.843550 23745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "5bb209b212104e4bb55097033bc47606"
format_stamp: "Formatted at 2026-06-18 22:29:53 on dist-test-slave-0956"
server_key: "35373fd5a3f63192ee4594bb7c91be6b"
server_key_iv: "91c4fa1e115dbb0ffcc03df386ab30a6"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:53.844659 23745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "5bb209b212104e4bb55097033bc47606"
format_stamp: "Formatted at 2026-06-18 22:29:53 on dist-test-slave-0956"
server_key: "35373fd5a3f63192ee4594bb7c91be6b"
server_key_iv: "91c4fa1e115dbb0ffcc03df386ab30a6"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:53.855072 23745 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.012s sys 0.001s
I20260618 22:29:53.885805 23769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:53.887984 23745 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.010s sys 0.003s
I20260618 22:29:53.888216 23745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "5bb209b212104e4bb55097033bc47606"
format_stamp: "Formatted at 2026-06-18 22:29:53 on dist-test-slave-0956"
server_key: "35373fd5a3f63192ee4594bb7c91be6b"
server_key_iv: "91c4fa1e115dbb0ffcc03df386ab30a6"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:53.888500 23745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:29:53.934509 23745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:29:53.939385 23745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:29:53.939682 23745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:29:53.940837 23745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:29:53.942505 23745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:29:53.942592 23745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:53.942700 23745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:29:53.942760 23745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:53.982568 23745 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:37359
I20260618 22:29:53.982578 23881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:37359 every 8 connection(s)
I20260618 22:29:53.984151 23745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:29:53.988415 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 23745
I20260618 22:29:53.988570 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:29:53.989064 11211 external_mini_cluster.cc:1428] Setting key 1f1d15ff89dc1bb8c46fbe9156bb9441
Jun 18 22:29:54 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821793, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:29:54.014133 23885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:53.989837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:52233 (local address 127.10.242.254:41203)
0618 22:29:53.992192 (+ 2355us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:53.992198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:53.992217 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:29:53.992271 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:53.992275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:53.995920 (+ 3645us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:53.996041 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:53.998179 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:53.999263 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:54.001676 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:54.002281 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:54.002381 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:54.007051 (+ 4670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:54.007060 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:54.007065 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:54.007110 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:54.010427 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:54.011384 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:54.011389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:54.011393 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:54.011540 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:54.012031 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:54.012036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:54.012040 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:54.012401 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:54.012520 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:54.013516 (+ 996us) server_negotiation.cc:299] Negotiation successful
0618 22:29:54.013758 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2224,"thread_start_us":2117,"threads_started":1}
I20260618 22:29:54.016558 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:53.987625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:41203 (local address 127.10.242.194:52233)
0618 22:29:53.988445 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:53.988500 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:53.990919 (+ 2419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:53.996486 (+ 5567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:53.996499 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:53.997125 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:29:53.997964 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:53.997991 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:53.999509 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:53.999514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:54.001507 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:54.001522 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:54.002954 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:54.002960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:54.003240 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:54.004028 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:29:54.004058 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:29:54.006748 (+ 2690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:29:54.010624 (+ 3876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:29:54.010633 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:29:54.010653 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:29:54.011244 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:29:54.011672 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:29:54.011677 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:29:54.011680 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:29:54.011914 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:29:54.012607 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:29:54.012615 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:29:54.013052 (+ 437us) client_negotiation.cc:769] Sending connection context
0618 22:29:54.014607 (+ 1555us) client_negotiation.cc:240] Negotiation successful
0618 22:29:54.015188 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":2310,"threads_started":1}
WARNING: no policy specified for kudu/127.10.242.195@KRBTEST.COM; defaulting to no policy
I20260618 22:29:54.018572 23882 heartbeater.cc:347] Connected to a master server at 127.10.242.254:41203
I20260618 22:29:54.018935 23882 heartbeater.cc:464] Registering TS with master...
I20260618 22:29:54.019870 23882 heartbeater.cc:511] Master 127.10.242.254:41203 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.195@KRBTEST.COM" created.
I20260618 22:29:54.021770 23526 ts_manager.cc:194] Registered new tserver with Master: 5bb209b212104e4bb55097033bc47606 (127.10.242.194:37359)
I20260618 22:29:54.022853 23526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:52233
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
I20260618 22:29:54.053619 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.212:34423/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:41203
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:29:54.223865 23739 heartbeater.cc:503] Master 127.10.242.254:41203 was elected leader, sending a full tablet report...
W20260618 22:29:54.397437 23888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:29:54.397717 23888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:29:54.397861 23888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:29:54.405117 23888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:29:54.405246 23888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:29:54.405397 23888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:29:54.415052 23888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.242.212:34423/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:41203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:29:54.416363 23888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:29:54.417689 23888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:29:54.430243 23893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:54.430222 23894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:54.430965 23896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:29:54.431421 23888 server_base.cc:1034] running on GCE node
I20260618 22:29:54.431978 23888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:29:54.432926 23888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:29:54.434151 23888 hybrid_clock.cc:648] HybridClock initialized: now 1781821794434055 us; error 91 us; skew 500 ppm
I20260618 22:29:54.434470 23888 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:29:54 dist-test-slave-0956 krb5kdc[23050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821794, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:29:54.437973 23888 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:29:54.439646 23888 webserver.cc:460] Webserver started at http://127.10.242.195:38069/ using document root <none> and password file <none>
I20260618 22:29:54.440461 23888 fs_manager.cc:359] Metadata directory not provided
I20260618 22:29:54.440582 23888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:29:54.440899 23888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:29:54 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821794, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
I20260618 22:29:54.458554 23888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "c3da7f7dd91441b9aecc72bb24f18d95"
format_stamp: "Formatted at 2026-06-18 22:29:54 on dist-test-slave-0956"
server_key: "af081e097d013f2c78d650e6dd7adaa3"
server_key_iv: "533aa6a2e677018cc2f3b1ea3163c99e"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:54.459451 23888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "c3da7f7dd91441b9aecc72bb24f18d95"
format_stamp: "Formatted at 2026-06-18 22:29:54 on dist-test-slave-0956"
server_key: "af081e097d013f2c78d650e6dd7adaa3"
server_key_iv: "533aa6a2e677018cc2f3b1ea3163c99e"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:54.465319 23888 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260618 22:29:54.482038 23903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:54.483951 23888 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260618 22:29:54.484200 23888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "c3da7f7dd91441b9aecc72bb24f18d95"
format_stamp: "Formatted at 2026-06-18 22:29:54 on dist-test-slave-0956"
server_key: "af081e097d013f2c78d650e6dd7adaa3"
server_key_iv: "533aa6a2e677018cc2f3b1ea3163c99e"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:54.484530 23888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:29:54.512097 23888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:29:54.516059 23888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:29:54.516340 23888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:29:54.517292 23888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:29:54.518677 23888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:29:54.518749 23888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:54.518852 23888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:29:54.518898 23888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:54.548267 23888 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:39635
I20260618 22:29:54.548309 24015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:39635 every 8 connection(s)
I20260618 22:29:54.549568 23888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:29:54.550930 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 23888
I20260618 22:29:54.551072 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:29:54.551497 11211 external_mini_cluster.cc:1428] Setting key 85223423572b150652fc7accf750f089
Jun 18 22:29:54 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821794, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:29:54.571218 24018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:54.552075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:49471 (local address 127.10.242.254:41203)
0618 22:29:54.552542 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:54.552548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:54.554799 (+ 2251us) server_negotiation.cc:407] Connection header received
0618 22:29:54.556592 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:54.556597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:54.556696 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:54.556789 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:54.558565 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:54.559290 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:54.560907 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:54.561416 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:54.561529 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:54.564826 (+ 3297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:54.564831 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:54.564835 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:54.564870 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:54.568049 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:54.568794 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:54.568798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:54.568800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:54.568934 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:54.569462 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:54.569468 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:54.569478 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:54.569754 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:54.569882 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:54.570645 (+ 763us) server_negotiation.cc:299] Negotiation successful
0618 22:29:54.570878 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":229,"threads_started":1}
I20260618 22:29:54.572222 24019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:54.552770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:41203 (local address 127.10.242.195:49471)
0618 22:29:54.554027 (+ 1257us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:54.554073 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:54.556305 (+ 2232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:54.557230 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:54.557241 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:54.557715 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:29:54.558380 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:54.558400 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:54.559444 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:54.559450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:54.560764 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:54.560776 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:54.561642 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:54.561650 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:54.561892 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:54.562506 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:29:54.562548 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:29:54.564590 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:29:54.568189 (+ 3599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:29:54.568196 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:29:54.568210 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:29:54.568634 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:29:54.569051 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:29:54.569055 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:29:54.569058 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:29:54.569353 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:29:54.569878 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:29:54.569886 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:29:54.570268 (+ 382us) client_negotiation.cc:769] Sending connection context
0618 22:29:54.570508 (+ 240us) client_negotiation.cc:240] Negotiation successful
0618 22:29:54.571052 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"thread_start_us":730,"threads_started":1}
I20260618 22:29:54.573660 24016 heartbeater.cc:347] Connected to a master server at 127.10.242.254:41203
I20260618 22:29:54.573966 24016 heartbeater.cc:464] Registering TS with master...
I20260618 22:29:54.574652 24016 heartbeater.cc:511] Master 127.10.242.254:41203 requested a full tablet report, sending...
I20260618 22:29:54.576109 23526 ts_manager.cc:194] Registered new tserver with Master: c3da7f7dd91441b9aecc72bb24f18d95 (127.10.242.195:39635)
I20260618 22:29:54.576936 23526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:49471
I20260618 22:29:54.580096 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:29:54.581750 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 23606
Jun 18 22:29:54 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:29:54.598657 24021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:54.582738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46214 (local address 127.10.242.193:41063)
0618 22:29:54.583337 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:54.583343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:54.583371 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:29:54.583484 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:54.583491 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:54.583705 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:54.583838 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:54.585295 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:54.586316 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:54.587895 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:54.588521 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:54.588627 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:54.592975 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:54.592989 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:54.593004 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:54.593041 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:54.596073 (+ 3032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:54.596778 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:54.596785 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:54.596787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:54.596915 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:54.597301 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:54.597304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:54.597306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:54.597618 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:54.597874 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:54.598109 (+ 235us) server_negotiation.cc:299] Negotiation successful
0618 22:29:54.598308 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":281,"threads_started":1}
I20260618 22:29:54.599969 23734 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:29:54.681247 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 23606
I20260618 22:29:54.693672 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 23745
Jun 18 22:29:54 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.194@KRBTEST.COM
I20260618 22:29:54.710325 24023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:54.694574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49204 (local address 127.10.242.194:37359)
0618 22:29:54.695123 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:54.695130 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:54.695158 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:29:54.695216 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:54.695222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:54.695446 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:54.695590 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:54.696833 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:54.697857 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:54.699188 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:54.699753 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:54.699867 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:54.704093 (+ 4226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:54.704105 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:54.704120 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:54.704167 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:54.707124 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:54.707970 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:54.707978 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:54.707980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:54.708129 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:54.708531 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:54.708535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:54.708538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:54.708900 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:54.709247 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:54.709605 (+ 358us) server_negotiation.cc:299] Negotiation successful
0618 22:29:54.709878 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":308,"threads_started":1}
I20260618 22:29:54.711730 23877 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:29:54.793865 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 23745
I20260618 22:29:54.805766 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 23888
Jun 18 22:29:54 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.195@KRBTEST.COM
I20260618 22:29:54.821799 24025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:54.806780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53074 (local address 127.10.242.195:39635)
0618 22:29:54.807371 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:54.807401 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:54.807429 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:29:54.807484 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:54.807490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:54.807697 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:54.807835 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:54.809197 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:54.810189 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:54.811527 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:54.812082 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:54.812192 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:54.816592 (+ 4400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:54.816607 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:54.816622 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:54.816665 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:54.819274 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:54.819927 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:54.819933 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:54.819936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:54.820052 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:54.820476 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:54.820479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:54.820481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:54.820794 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:54.821027 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:54.821324 (+ 297us) server_negotiation.cc:299] Negotiation successful
0618 22:29:54.821532 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":308,"threads_started":1}
I20260618 22:29:54.823145 24011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:29:54.906944 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 23888
I20260618 22:29:54.919615 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 23494
I20260618 22:29:54.920924 23556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:29:55.032068 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 23494
I20260618 22:29:55.048936 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:41203
--webserver_interface=127.10.242.254
--webserver_port=34123
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:41203
--ranger_config_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.212:34423/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.242.212:34423/kms with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:29:55.365837 24028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:29:55.366127 24028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:29:55.366220 24028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:29:55.373637 24028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:29:55.373765 24028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:29:55.373811 24028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:29:55.373864 24028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:29:55.373908 24028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:29:55.384089 24028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.242.212:34423/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:41203
--ranger_config_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:41203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=34123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:29:55.385386 24028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:29:55.386795 24028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:29:55.398559 24033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:55.398602 24034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:55.398629 24036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:29:55.399127 24028 server_base.cc:1034] running on GCE node
I20260618 22:29:55.399910 24028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:29:55.401345 24028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:29:55.402542 24028 hybrid_clock.cc:648] HybridClock initialized: now 1781821795402496 us; error 39 us; skew 500 ppm
I20260618 22:29:55.402894 24028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:29:55 dist-test-slave-0956 krb5kdc[23050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821795, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:29:55.406193 24028 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:29:55.407785 24028 webserver.cc:460] Webserver started at http://127.10.242.254:34123/ using document root <none> and password file <none>
I20260618 22:29:55.408504 24028 fs_manager.cc:359] Metadata directory not provided
I20260618 22:29:55.408627 24028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 18 22:29:55 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821795, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
I20260618 22:29:55.495015 24028 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260618 22:29:55.498179 24044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:55.499562 24028 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:29:55.499737 24028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "fabb7980629643faaa413a42d1b5d1f2"
format_stamp: "Formatted at 2026-06-18 22:29:51 on dist-test-slave-0956"
server_key: "f309f646f7d1bcf1c874b35340376f4f"
server_key_iv: "3544a8c2e72528bde08e0fa16e1528fb"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:55.500716 24028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:29:55.515661 24028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:29:55.519692 24028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:29:55.519970 24028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:29:55.536613 24028 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:41203
I20260618 22:29:55.536620 24095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:41203 every 8 connection(s)
I20260618 22:29:55.538072 24028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:29:55.540917 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 24028
I20260618 22:29:55.541743 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.212:34423/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:41063
--local_ip_for_outbound_sockets=127.10.242.193
--tserver_master_addrs=127.10.242.254:41203
--webserver_port=34777
--webserver_interface=127.10.242.193
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.242.212:34423/kms with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:29:55.544998 24096 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:29:55.548331 24096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2: Bootstrap starting.
I20260618 22:29:55.568280 24096 log.cc:824] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2: Log is configured to *not* fsync() on all Append() calls
I20260618 22:29:55.577280 24096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:29:55.577782 24096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2: Bootstrap complete.
I20260618 22:29:55.586465 24096 raft_consensus.cc:348] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabb7980629643faaa413a42d1b5d1f2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 41203 } }
I20260618 22:29:55.587291 24096 raft_consensus.cc:729] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb7980629643faaa413a42d1b5d1f2, State: Initialized, Role: FOLLOWER
I20260618 22:29:55.587919 24096 consensus_queue.cc:260] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabb7980629643faaa413a42d1b5d1f2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 41203 } }
I20260618 22:29:55.588122 24096 raft_consensus.cc:388] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:29:55.588229 24096 raft_consensus.cc:482] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:29:55.588379 24096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:29:55.592365 24096 raft_consensus.cc:504] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabb7980629643faaa413a42d1b5d1f2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 41203 } }
I20260618 22:29:55.592989 24096 leader_election.cc:302] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fabb7980629643faaa413a42d1b5d1f2; no voters:
I20260618 22:29:55.593609 24096 leader_election.cc:288] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260618 22:29:55.593850 24100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260618 22:29:55.594705 24100 raft_consensus.cc:686] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [term 2 LEADER]: Becoming Leader. State: Replica: fabb7980629643faaa413a42d1b5d1f2, State: Running, Role: LEADER
I20260618 22:29:55.595359 24100 consensus_queue.cc:237] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabb7980629643faaa413a42d1b5d1f2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 41203 } }
I20260618 22:29:55.595710 24096 sys_catalog.cc:564] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:29:55.597944 24102 sys_catalog.cc:455] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fabb7980629643faaa413a42d1b5d1f2. Latest consensus state: current_term: 2 leader_uuid: "fabb7980629643faaa413a42d1b5d1f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabb7980629643faaa413a42d1b5d1f2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 41203 } } }
I20260618 22:29:55.598364 24102 sys_catalog.cc:458] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [sys.catalog]: This master's current role is: LEADER
I20260618 22:29:55.598910 24101 sys_catalog.cc:455] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fabb7980629643faaa413a42d1b5d1f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabb7980629643faaa413a42d1b5d1f2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 41203 } } }
I20260618 22:29:55.599218 24101 sys_catalog.cc:458] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2 [sys.catalog]: This master's current role is: LEADER
I20260618 22:29:55.602429 24106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:29:55.612542 24106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:29:55.613380 24096 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260618 22:29:55.614207 24106 catalog_manager.cc:1259] Loaded cluster ID: db430e83521d46b5814e3442171a21ae
I20260618 22:29:55.614382 24106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:29:55.618364 24106 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:29:55.619901 24106 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2: Loaded TSK: 0
I20260618 22:29:55.621613 24106 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260618 22:29:55.958671 24098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:29:55.959177 24098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:29:55.959394 24098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:29:55.970867 24098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:29:55.971017 24098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:29:55.971257 24098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:29:55.986490 24098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.242.212:34423/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:41063
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=34777
--tserver_master_addrs=127.10.242.254:41203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:29:55.988345 24098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:29:55.990207 24098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:29:56.015003 24134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:56.016274 24135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:56.025542 24137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:29:56.026567 24098 server_base.cc:1034] running on GCE node
I20260618 22:29:56.027671 24098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:29:56.033912 24098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:29:56.035274 24098 hybrid_clock.cc:648] HybridClock initialized: now 1781821796035141 us; error 124 us; skew 500 ppm
I20260618 22:29:56.035771 24098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:29:56 dist-test-slave-0956 krb5kdc[23050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821796, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:29:56.043392 24098 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:29:56.048158 24098 webserver.cc:460] Webserver started at http://127.10.242.193:34777/ using document root <none> and password file <none>
I20260618 22:29:56.049317 24098 fs_manager.cc:359] Metadata directory not provided
I20260618 22:29:56.049527 24098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 18 22:29:56 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821796, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
I20260618 22:29:56.100066 24098 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.007s sys 0.000s
I20260618 22:29:56.104087 24145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:56.106077 24098 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260618 22:29:56.106268 24098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "ae65410218aa462ca85d19dc8dc779e9"
format_stamp: "Formatted at 2026-06-18 22:29:52 on dist-test-slave-0956"
server_key: "7104310e9977ac2c39fcc6744896ffec"
server_key_iv: "4aef98afd06321e8f41f0edfc332a593"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:56.107718 24098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:29:56.147358 24098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:29:56.155123 24098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:29:56.155483 24098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:29:56.156773 24098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:29:56.158685 24098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:29:56.158812 24098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:56.158932 24098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:29:56.159014 24098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:56.243934 24098 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:41063
I20260618 22:29:56.243957 24258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:41063 every 8 connection(s)
I20260618 22:29:56.245975 24098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:29:56.254966 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 24098
I20260618 22:29:56.255756 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.212:34423/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:37359
--local_ip_for_outbound_sockets=127.10.242.194
--tserver_master_addrs=127.10.242.254:41203
--webserver_port=42083
--webserver_interface=127.10.242.194
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.242.212:34423/kms with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:29:56 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821796, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:29:56.285113 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:56.249377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:41203 (local address 127.10.242.193:43823)
0618 22:29:56.250205 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:56.250260 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:56.252107 (+ 1847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:56.263257 (+ 11150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:56.263274 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:56.263992 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:29:56.264718 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:56.264740 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:56.267232 (+ 2492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:56.267238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:56.269137 (+ 1899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:56.269156 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:56.270768 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:56.270774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:56.271099 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:56.271912 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:29:56.271949 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:29:56.274870 (+ 2921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:29:56.279574 (+ 4704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:29:56.279584 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:29:56.279602 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:29:56.280232 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:29:56.280844 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:29:56.280849 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:29:56.280853 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:29:56.281174 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:29:56.282354 (+ 1180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:29:56.282362 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:29:56.282815 (+ 453us) client_negotiation.cc:769] Sending connection context
0618 22:29:56.283147 (+ 332us) client_negotiation.cc:240] Negotiation successful
0618 22:29:56.283734 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":323,"threads_started":1}
I20260618 22:29:56.285123 24262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:56.249843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:43823 (local address 127.10.242.254:41203)
0618 22:29:56.251041 (+ 1198us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:56.251058 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:56.251112 (+ 54us) server_negotiation.cc:407] Connection header received
0618 22:29:56.252745 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:56.252760 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:56.253249 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:56.253703 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:56.265019 (+ 11316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:56.266983 (+ 1964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:56.269329 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:56.270494 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:56.270724 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:56.275216 (+ 4492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:56.275227 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:56.275248 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:56.275307 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:56.279371 (+ 4064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:56.280428 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:56.280439 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:56.280449 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:56.280678 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:56.281327 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:56.281332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:56.281336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:56.282074 (+ 738us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:56.282260 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:56.283407 (+ 1147us) server_negotiation.cc:299] Negotiation successful
0618 22:29:56.283821 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":949,"thread_start_us":744,"threads_started":1}
I20260618 22:29:56.289536 24259 heartbeater.cc:347] Connected to a master server at 127.10.242.254:41203
I20260618 22:29:56.289920 24259 heartbeater.cc:464] Registering TS with master...
I20260618 22:29:56.290835 24259 heartbeater.cc:511] Master 127.10.242.254:41203 requested a full tablet report, sending...
W20260618 22:29:56.294279 24259 heartbeater.cc:644] Failed to heartbeat to 127.10.242.254:41203 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260618 22:29:56.294523 24259 heartbeater.cc:464] Registering TS with master...
I20260618 22:29:56.294821 24259 heartbeater.cc:511] Master 127.10.242.254:41203 requested a full tablet report, sending...
I20260618 22:29:56.296118 24259 heartbeater.cc:464] Registering TS with master...
I20260618 22:29:56.296430 24259 heartbeater.cc:511] Master 127.10.242.254:41203 requested a full tablet report, sending...
W20260618 22:29:56.297523 24259 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260618 22:29:56.638402 24263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:29:56.638712 24263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:29:56.638864 24263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:29:56.646966 24263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:29:56.647104 24263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:29:56.647261 24263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:29:56.657766 24263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.242.212:34423/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:37359
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=42083
--tserver_master_addrs=127.10.242.254:41203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:29:56.659139 24263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:29:56.660444 24263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:29:56.681725 24277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:56.683171 24278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:56.684046 24280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:29:56.684391 24263 server_base.cc:1034] running on GCE node
I20260618 22:29:56.685354 24263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:29:56.686691 24263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:29:56.687960 24263 hybrid_clock.cc:648] HybridClock initialized: now 1781821796687887 us; error 57 us; skew 500 ppm
I20260618 22:29:56.688437 24263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:29:56 dist-test-slave-0956 krb5kdc[23050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821796, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:29:56.692459 24263 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:29:56.694343 24263 webserver.cc:460] Webserver started at http://127.10.242.194:42083/ using document root <none> and password file <none>
I20260618 22:29:56.695258 24263 fs_manager.cc:359] Metadata directory not provided
I20260618 22:29:56.695423 24263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 18 22:29:56 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821796, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
I20260618 22:29:56.736161 24263 fs_manager.cc:709] Time spent opening directory manager: real 0.017s user 0.009s sys 0.008s
I20260618 22:29:56.742000 24290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:56.743880 24263 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260618 22:29:56.744093 24263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "5bb209b212104e4bb55097033bc47606"
format_stamp: "Formatted at 2026-06-18 22:29:53 on dist-test-slave-0956"
server_key: "35373fd5a3f63192ee4594bb7c91be6b"
server_key_iv: "91c4fa1e115dbb0ffcc03df386ab30a6"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:56.745702 24263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:29:56.783226 24263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:29:56.787417 24263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:29:56.787676 24263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:29:56.788734 24263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:29:56.790222 24263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:29:56.790316 24263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:56.790422 24263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:29:56.790491 24263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:29:56 dist-test-slave-0956 krb5kdc[23050](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781821796, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:29:56.865432 24263 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:37359
I20260618 22:29:56.865541 24402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:37359 every 8 connection(s)
I20260618 22:29:56.867411 24263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:29:56.877156 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 24263
I20260618 22:29:56.877946 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.212:34423/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:39635
--local_ip_for_outbound_sockets=127.10.242.195
--tserver_master_addrs=127.10.242.254:41203
--webserver_port=38069
--webserver_interface=127.10.242.195
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.242.212:34423/kms with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:29:56 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821796, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:29:56.904232 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:56.871986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:42279 (local address 127.10.242.254:41203)
0618 22:29:56.884641 (+ 12655us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:56.884647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:56.884673 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:29:56.884740 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:56.884746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:56.884829 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:56.884958 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:56.887204 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:56.888469 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:56.890938 (+ 2469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:56.891542 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:56.891682 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:56.896181 (+ 4499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:56.896190 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:56.896195 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:56.896243 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:56.899795 (+ 3552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:56.900849 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:56.900855 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:56.900859 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:56.901021 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:56.901648 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:56.901653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:56.901656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:56.902047 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:56.902202 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:56.903532 (+ 1330us) server_negotiation.cc:299] Negotiation successful
0618 22:29:56.903815 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12510,"thread_start_us":12396,"threads_started":1}
I20260618 22:29:56.905254 24405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:56.871394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:41203 (local address 127.10.242.194:42279)
0618 22:29:56.874561 (+ 3167us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:56.874617 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:56.878165 (+ 3548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:56.885369 (+ 7204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:56.885384 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:56.886066 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:29:56.886968 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:56.886997 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:56.888668 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:56.888674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:56.890740 (+ 2066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:56.890757 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:56.892005 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:56.892011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:56.892286 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:56.893123 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:29:56.893157 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:29:56.895866 (+ 2709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:29:56.900001 (+ 4135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:29:56.900011 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:29:56.900032 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:29:56.900671 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:29:56.901204 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:29:56.901209 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:29:56.901213 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:29:56.901499 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:29:56.902292 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:29:56.902301 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:29:56.902810 (+ 509us) client_negotiation.cc:769] Sending connection context
0618 22:29:56.903137 (+ 327us) client_negotiation.cc:240] Negotiation successful
0618 22:29:56.903730 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2838,"thread_start_us":2606,"threads_started":1}
I20260618 22:29:56.906834 24403 heartbeater.cc:347] Connected to a master server at 127.10.242.254:41203
I20260618 22:29:56.907099 24403 heartbeater.cc:464] Registering TS with master...
I20260618 22:29:56.907799 24403 heartbeater.cc:511] Master 127.10.242.254:41203 requested a full tablet report, sending...
I20260618 22:29:56.909689 24061 ts_manager.cc:194] Registered new tserver with Master: 5bb209b212104e4bb55097033bc47606 (127.10.242.194:37359)
I20260618 22:29:56.911808 24061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:42279
Jun 18 22:29:57 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821796, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
I20260618 22:29:57.297675 24259 heartbeater.cc:464] Registering TS with master...
I20260618 22:29:57.298080 24259 heartbeater.cc:511] Master 127.10.242.254:41203 requested a full tablet report, sending...
I20260618 22:29:57.299206 24061 ts_manager.cc:194] Registered new tserver with Master: ae65410218aa462ca85d19dc8dc779e9 (127.10.242.193:41063)
I20260618 22:29:57.300016 24061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:43823
W20260618 22:29:57.363844 24407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:29:57.364279 24407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:29:57.364437 24407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:29:57.375413 24407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:29:57.375545 24407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:29:57.375698 24407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:29:57.385628 24407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.242.212:34423/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:39635
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=38069
--tserver_master_addrs=127.10.242.254:41203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:29:57.386952 24407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:29:57.388286 24407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:29:57.404939 24414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:57.405049 24413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:29:57.405926 24416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:29:57.407161 24407 server_base.cc:1034] running on GCE node
I20260618 22:29:57.407877 24407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:29:57.409022 24407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:29:57.410269 24407 hybrid_clock.cc:648] HybridClock initialized: now 1781821797410202 us; error 52 us; skew 500 ppm
I20260618 22:29:57.410620 24407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:29:57 dist-test-slave-0956 krb5kdc[23050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821797, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:29:57.414515 24407 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:29:57.424002 24407 webserver.cc:460] Webserver started at http://127.10.242.195:38069/ using document root <none> and password file <none>
I20260618 22:29:57.424923 24407 fs_manager.cc:359] Metadata directory not provided
I20260618 22:29:57.425098 24407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 18 22:29:57 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821797, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
I20260618 22:29:57.461578 24407 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.010s sys 0.000s
I20260618 22:29:57.465392 24424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:57.467342 24407 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260618 22:29:57.467532 24407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "c3da7f7dd91441b9aecc72bb24f18d95"
format_stamp: "Formatted at 2026-06-18 22:29:54 on dist-test-slave-0956"
server_key: "af081e097d013f2c78d650e6dd7adaa3"
server_key_iv: "533aa6a2e677018cc2f3b1ea3163c99e"
server_key_version: "kuduclusterkey@0"
I20260618 22:29:57.468906 24407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:29:57.484452 24407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:29:57.489917 24407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:29:57.490243 24407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:29:57.492421 24407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:29:57.494236 24407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:29:57.494367 24407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:29:57.494483 24407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:29:57.494560 24407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:29:57 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821796, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for HTTP/127.10.242.212@KRBTEST.COM
I20260618 22:29:57.595903 24407 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:39635
I20260618 22:29:57.598040 24407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:29:57.598590 24537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:39635 every 8 connection(s)
I20260618 22:29:57.605556 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 24407
Jun 18 22:29:57 dist-test-slave-0956 krb5kdc[23050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821797, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:29:57.652114 24540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:57.606553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48170 (local address 127.10.242.254:41203)
0618 22:29:57.627602 (+ 21049us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:57.627608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:57.628001 (+ 393us) server_negotiation.cc:407] Connection header received
0618 22:29:57.628065 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:57.628069 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:57.628153 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:57.630859 (+ 2706us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:57.631066 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.632404 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.634443 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.635084 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.637990 (+ 2906us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:57.641753 (+ 3763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:57.641761 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:57.641766 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:57.641806 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:57.647281 (+ 5475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:57.648973 (+ 1692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:57.648978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:57.648981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:57.649157 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:57.649661 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:57.649667 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:57.649670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:57.650034 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:57.650302 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:57.651472 (+ 1170us) server_negotiation.cc:299] Negotiation successful
0618 22:29:57.651726 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":20931,"thread_start_us":19542,"threads_started":1}
I20260618 22:29:57.655056 24541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:57.604712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:41203 (local address 127.10.242.195:54251)
0618 22:29:57.618966 (+ 14254us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:57.619022 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:57.620863 (+ 1841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:57.627084 (+ 6221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:57.627099 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:57.627858 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:29:57.628848 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:57.628880 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.630424 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.630429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:57.632552 (+ 2123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:57.632567 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.639195 (+ 6628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.639203 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:57.639548 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:57.640294 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:29:57.640328 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:29:57.643023 (+ 2695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:29:57.649016 (+ 5993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:29:57.649027 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:29:57.649049 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:29:57.649754 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:29:57.650213 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:29:57.650217 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:29:57.650220 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:29:57.650599 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:29:57.651901 (+ 1302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:29:57.651910 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:29:57.652691 (+ 781us) client_negotiation.cc:769] Sending connection context
0618 22:29:57.653011 (+ 320us) client_negotiation.cc:240] Negotiation successful
0618 22:29:57.653664 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13908,"thread_start_us":13677,"threads_started":1}
I20260618 22:29:57.655699 24543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:57.603963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:54251 (local address 127.10.242.254:41203)
0618 22:29:57.626372 (+ 22409us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:57.626379 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:57.626401 (+ 22us) server_negotiation.cc:407] Connection header received
0618 22:29:57.626467 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:57.626472 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:57.626568 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:57.626687 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:57.629075 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.630213 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.638219 (+ 8006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.638923 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.639972 (+ 1049us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:57.643517 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:57.643524 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:57.643529 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:57.643581 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:57.648827 (+ 5246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:57.649909 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:57.649914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:57.649917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:57.650060 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:57.651267 (+ 1207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:57.651282 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:57.651286 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:57.651749 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:57.654945 (+ 3196us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:57.655093 (+ 148us) server_negotiation.cc:299] Negotiation successful
0618 22:29:57.655342 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22256,"thread_start_us":23367,"threads_started":1}
I20260618 22:29:57.657064 24538 heartbeater.cc:347] Connected to a master server at 127.10.242.254:41203
I20260618 22:29:57.657423 24538 heartbeater.cc:464] Registering TS with master...
I20260618 22:29:57.658317 24538 heartbeater.cc:511] Master 127.10.242.254:41203 requested a full tablet report, sending...
I20260618 22:29:57.660036 24061 ts_manager.cc:194] Registered new tserver with Master: c3da7f7dd91441b9aecc72bb24f18d95 (127.10.242.195:39635)
I20260618 22:29:57.660878 24061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:54251
I20260618 22:29:57.673765 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:29:57.748229 24543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:57.714870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48184 (local address 127.10.242.254:41203)
0618 22:29:57.715073 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:57.715081 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:57.730187 (+ 15106us) server_negotiation.cc:407] Connection header received
0618 22:29:57.730795 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:57.730803 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:57.730903 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:57.730980 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:57.733354 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.734494 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.736559 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.737260 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.737389 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:57.741030 (+ 3641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:57.741039 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:57.741044 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:57.741131 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:57.744618 (+ 3487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:57.745683 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:57.745689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:57.745693 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:57.745865 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:57.746347 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:57.746352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:57.746355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:57.746739 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:57.747001 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:57.747613 (+ 612us) server_negotiation.cc:299] Negotiation successful
0618 22:29:57.747883 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260618 22:29:57.755626 24061 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48184:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:29:57.767314 24061 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:29:57.835572 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:57.796474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:39635 (local address 127.0.0.1:43678)
0618 22:29:57.812422 (+ 15948us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:57.812492 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:57.812748 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:57.819102 (+ 6354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:57.819112 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:57.819176 (+ 64us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:29:57.819608 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:57.819622 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.829831 (+ 10209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.829838 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:57.832215 (+ 2377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:57.832238 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.834591 (+ 2353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.834598 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:57.834803 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:57.834869 (+ 66us) client_negotiation.cc:769] Sending connection context
0618 22:29:57.835005 (+ 136us) client_negotiation.cc:240] Negotiation successful
0618 22:29:57.835205 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15560,"thread_start_us":16506,"threads_started":1}
I20260618 22:29:57.836651 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:57.796751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43678 (local address 127.10.242.195:39635)
0618 22:29:57.817718 (+ 20967us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:57.817729 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:57.817777 (+ 48us) server_negotiation.cc:407] Connection header received
0618 22:29:57.817909 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:57.817920 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:57.818245 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:57.818512 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:29:57.819879 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.829621 (+ 9742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.832450 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.834055 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.834243 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:57.834441 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:57.836115 (+ 1674us) server_negotiation.cc:299] Negotiation successful
0618 22:29:57.836300 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20735,"thread_start_us":21990,"threads_started":1}
I20260618 22:29:57.841233 24557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:57.820368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:37359 (local address 127.0.0.1:40080)
0618 22:29:57.828578 (+ 8210us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:57.828602 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:57.828759 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:57.832384 (+ 3625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:57.832389 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:57.832423 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:29:57.832737 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:57.832748 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.835975 (+ 3227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.835980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:57.838177 (+ 2197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:57.838193 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.840457 (+ 2264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.840464 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:57.840627 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:57.840661 (+ 34us) client_negotiation.cc:769] Sending connection context
0618 22:29:57.840745 (+ 84us) client_negotiation.cc:240] Negotiation successful
0618 22:29:57.840889 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8049,"thread_start_us":8483,"threads_started":1}
I20260618 22:29:57.840937 24472 tablet_service.cc:1467] Processing CreateTablet for tablet 835fde719ad8498ab26bee5cf44bafe8 (DEFAULT_TABLE table=test-table [id=d7e855275be34935a8ed7e620c35188c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:29:57.842511 24472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 835fde719ad8498ab26bee5cf44bafe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:29:57.848953 24556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:57.814280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40080 (local address 127.10.242.194:37359)
0618 22:29:57.831055 (+ 16775us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:57.831065 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:57.831099 (+ 34us) server_negotiation.cc:407] Connection header received
0618 22:29:57.831211 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:57.831220 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:57.831500 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:57.831716 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:29:57.834055 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.835530 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.838594 (+ 3064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.839990 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.840146 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:57.840310 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:57.843928 (+ 3618us) server_negotiation.cc:299] Negotiation successful
0618 22:29:57.844115 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16590,"thread_start_us":17594,"threads_started":1}
I20260618 22:29:57.853650 24338 tablet_service.cc:1467] Processing CreateTablet for tablet 835fde719ad8498ab26bee5cf44bafe8 (DEFAULT_TABLE table=test-table [id=d7e855275be34935a8ed7e620c35188c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:29:57.855355 24338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 835fde719ad8498ab26bee5cf44bafe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:29:57.858907 24558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:57.829289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:41063 (local address 127.0.0.1:39530)
0618 22:29:57.833573 (+ 4284us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:57.833592 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:57.833703 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:57.845465 (+ 11762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:57.845470 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:57.845498 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:29:57.845790 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:57.845800 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.852326 (+ 6526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.852333 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:57.854387 (+ 2054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:57.854520 (+ 133us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.858081 (+ 3561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.858088 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:57.858283 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:57.858319 (+ 36us) client_negotiation.cc:769] Sending connection context
0618 22:29:57.858393 (+ 74us) client_negotiation.cc:240] Negotiation successful
0618 22:29:57.858548 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4136,"thread_start_us":4474,"threads_started":1}
I20260618 22:29:57.867033 24559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:57.831629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39530 (local address 127.10.242.193:41063)
0618 22:29:57.843308 (+ 11679us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:57.843317 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:57.843346 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:29:57.843445 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:57.843453 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:57.843692 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:57.846015 (+ 2323us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:29:57.846260 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.850920 (+ 4660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.854750 (+ 3830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:57.857837 (+ 3087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:57.859550 (+ 1713us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:57.866310 (+ 6760us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:57.866409 (+ 99us) server_negotiation.cc:299] Negotiation successful
0618 22:29:57.866632 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11502,"thread_start_us":29459,"threads_started":1}
I20260618 22:29:57.871726 24194 tablet_service.cc:1467] Processing CreateTablet for tablet 835fde719ad8498ab26bee5cf44bafe8 (DEFAULT_TABLE table=test-table [id=d7e855275be34935a8ed7e620c35188c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:29:57.874223 24194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 835fde719ad8498ab26bee5cf44bafe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:29:57.893059 24560 tablet_bootstrap.cc:492] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606: Bootstrap starting.
I20260618 22:29:57.897830 24561 tablet_bootstrap.cc:492] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95: Bootstrap starting.
I20260618 22:29:57.899762 24560 tablet_bootstrap.cc:654] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606: Neither blocks nor log segments found. Creating new log.
I20260618 22:29:57.901211 24560 log.cc:824] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606: Log is configured to *not* fsync() on all Append() calls
I20260618 22:29:57.901479 24561 tablet_bootstrap.cc:654] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95: Neither blocks nor log segments found. Creating new log.
I20260618 22:29:57.902297 24562 tablet_bootstrap.cc:492] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9: Bootstrap starting.
I20260618 22:29:57.902733 24561 log.cc:824] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95: Log is configured to *not* fsync() on all Append() calls
I20260618 22:29:57.907208 24562 tablet_bootstrap.cc:654] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9: Neither blocks nor log segments found. Creating new log.
I20260618 22:29:57.908450 24562 log.cc:824] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9: Log is configured to *not* fsync() on all Append() calls
I20260618 22:29:57.909382 24561 tablet_bootstrap.cc:492] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95: No bootstrap required, opened a new log
I20260618 22:29:57.909693 24561 ts_tablet_manager.cc:1397] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260618 22:29:57.916046 24403 heartbeater.cc:503] Master 127.10.242.254:41203 was elected leader, sending a full tablet report...
I20260618 22:29:57.919011 24562 tablet_bootstrap.cc:492] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9: No bootstrap required, opened a new log
I20260618 22:29:57.919313 24562 ts_tablet_manager.cc:1397] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.000s
I20260618 22:29:57.920782 24561 raft_consensus.cc:348] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3da7f7dd91441b9aecc72bb24f18d95" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39635 } } peers { permanent_uuid: "5bb209b212104e4bb55097033bc47606" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37359 } } peers { permanent_uuid: "ae65410218aa462ca85d19dc8dc779e9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 41063 } }
I20260618 22:29:57.921276 24561 raft_consensus.cc:374] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:29:57.921391 24561 raft_consensus.cc:729] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3da7f7dd91441b9aecc72bb24f18d95, State: Initialized, Role: FOLLOWER
I20260618 22:29:57.922101 24561 consensus_queue.cc:260] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3da7f7dd91441b9aecc72bb24f18d95" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39635 } } peers { permanent_uuid: "5bb209b212104e4bb55097033bc47606" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37359 } } peers { permanent_uuid: "ae65410218aa462ca85d19dc8dc779e9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 41063 } }
I20260618 22:29:57.927829 24560 tablet_bootstrap.cc:492] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606: No bootstrap required, opened a new log
I20260618 22:29:57.928148 24560 ts_tablet_manager.cc:1397] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606: Time spent bootstrapping tablet: real 0.035s user 0.010s sys 0.010s
I20260618 22:29:57.931083 24562 raft_consensus.cc:348] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3da7f7dd91441b9aecc72bb24f18d95" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39635 } } peers { permanent_uuid: "5bb209b212104e4bb55097033bc47606" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37359 } } peers { permanent_uuid: "ae65410218aa462ca85d19dc8dc779e9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 41063 } }
I20260618 22:29:57.931553 24562 raft_consensus.cc:374] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:29:57.931649 24562 raft_consensus.cc:729] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae65410218aa462ca85d19dc8dc779e9, State: Initialized, Role: FOLLOWER
I20260618 22:29:57.932406 24562 consensus_queue.cc:260] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3da7f7dd91441b9aecc72bb24f18d95" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39635 } } peers { permanent_uuid: "5bb209b212104e4bb55097033bc47606" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37359 } } peers { permanent_uuid: "ae65410218aa462ca85d19dc8dc779e9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 41063 } }
I20260618 22:29:57.939808 24560 raft_consensus.cc:348] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3da7f7dd91441b9aecc72bb24f18d95" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39635 } } peers { permanent_uuid: "5bb209b212104e4bb55097033bc47606" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37359 } } peers { permanent_uuid: "ae65410218aa462ca85d19dc8dc779e9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 41063 } }
I20260618 22:29:57.940397 24560 raft_consensus.cc:374] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:29:57.940518 24560 raft_consensus.cc:729] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bb209b212104e4bb55097033bc47606, State: Initialized, Role: FOLLOWER
I20260618 22:29:57.944049 24538 heartbeater.cc:503] Master 127.10.242.254:41203 was elected leader, sending a full tablet report...
I20260618 22:29:57.945250 24561 ts_tablet_manager.cc:1428] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95: Time spent starting tablet: real 0.035s user 0.009s sys 0.011s
I20260618 22:29:57.946019 24560 consensus_queue.cc:260] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3da7f7dd91441b9aecc72bb24f18d95" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39635 } } peers { permanent_uuid: "5bb209b212104e4bb55097033bc47606" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37359 } } peers { permanent_uuid: "ae65410218aa462ca85d19dc8dc779e9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 41063 } }
I20260618 22:29:57.952795 24562 ts_tablet_manager.cc:1428] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9: Time spent starting tablet: real 0.033s user 0.013s sys 0.005s
I20260618 22:29:57.952883 24560 ts_tablet_manager.cc:1428] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606: Time spent starting tablet: real 0.025s user 0.019s sys 0.001s
I20260618 22:29:57.953331 24259 heartbeater.cc:503] Master 127.10.242.254:41203 was elected leader, sending a full tablet report...
W20260618 22:29:58.000159 24260 tablet.cc:2369] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:29:58.043098 24567 raft_consensus.cc:482] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:29:58.043381 24567 raft_consensus.cc:504] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3da7f7dd91441b9aecc72bb24f18d95" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39635 } } peers { permanent_uuid: "5bb209b212104e4bb55097033bc47606" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37359 } } peers { permanent_uuid: "ae65410218aa462ca85d19dc8dc779e9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 41063 } }
I20260618 22:29:58.045045 24567 leader_election.cc:288] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3da7f7dd91441b9aecc72bb24f18d95 (127.10.242.195:39635), 5bb209b212104e4bb55097033bc47606 (127.10.242.194:37359)
I20260618 22:29:58.053332 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:58.045525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:54313 (local address 127.10.242.195:39635)
0618 22:29:58.045684 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:58.045688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:58.046331 (+ 643us) server_negotiation.cc:407] Connection header received
0618 22:29:58.046741 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:58.046748 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:58.046849 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:58.046929 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:29:58.047852 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:58.048869 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:58.051154 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:58.052325 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:58.052493 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:58.052568 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:58.052909 (+ 341us) server_negotiation.cc:299] Negotiation successful
0618 22:29:58.053042 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260618 22:29:58.053735 24576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:58.045503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:39635 (local address 127.10.242.193:54313)
0618 22:29:58.046192 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:58.046229 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:58.046499 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:58.047201 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:58.047205 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:58.047233 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:29:58.047672 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:58.047685 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:58.049002 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:58.049007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:58.050963 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:58.050979 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:58.052552 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:58.052559 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:58.052732 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:58.052767 (+ 35us) client_negotiation.cc:769] Sending connection context
0618 22:29:58.052866 (+ 99us) client_negotiation.cc:240] Negotiation successful
0618 22:29:58.053001 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":50,"thread_start_us":321,"threads_started":1}
I20260618 22:29:58.054512 24577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:58.046310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:37359 (local address 127.10.242.193:45457)
0618 22:29:58.047765 (+ 1455us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:58.047783 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:29:58.047897 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:29:58.048188 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:29:58.048192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:29:58.048214 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:29:58.048513 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:58.048522 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:58.049789 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:58.049794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:58.051789 (+ 1995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:29:58.051804 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:29:58.053744 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:29:58.053750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:29:58.053915 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:58.053945 (+ 30us) client_negotiation.cc:769] Sending connection context
0618 22:29:58.054032 (+ 87us) client_negotiation.cc:240] Negotiation successful
0618 22:29:58.054160 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1342,"mutex_wait_us":1,"thread_start_us":1126,"threads_started":1}
I20260618 22:29:58.054816 24492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "835fde719ad8498ab26bee5cf44bafe8" candidate_uuid: "ae65410218aa462ca85d19dc8dc779e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3da7f7dd91441b9aecc72bb24f18d95" is_pre_election: true
I20260618 22:29:58.055286 24492 raft_consensus.cc:2454] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae65410218aa462ca85d19dc8dc779e9 in term 0.
I20260618 22:29:58.055789 24556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:58.046802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:45457 (local address 127.10.242.194:37359)
0618 22:29:58.046981 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:58.046987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:58.047849 (+ 862us) server_negotiation.cc:407] Connection header received
0618 22:29:58.048029 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:58.048033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:58.048091 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:58.048162 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:29:58.048665 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:58.049630 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:58.051982 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:58.053105 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:58.053287 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:58.053356 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:58.054063 (+ 707us) server_negotiation.cc:299] Negotiation successful
0618 22:29:58.054197 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:29:58.055950 24148 leader_election.cc:302] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae65410218aa462ca85d19dc8dc779e9, c3da7f7dd91441b9aecc72bb24f18d95; no voters:
I20260618 22:29:58.056382 24567 raft_consensus.cc:2781] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:29:58.056512 24567 raft_consensus.cc:482] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:29:58.056613 24567 raft_consensus.cc:3037] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:29:58.056690 24358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "835fde719ad8498ab26bee5cf44bafe8" candidate_uuid: "ae65410218aa462ca85d19dc8dc779e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bb209b212104e4bb55097033bc47606" is_pre_election: true
I20260618 22:29:58.057241 24358 raft_consensus.cc:2454] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae65410218aa462ca85d19dc8dc779e9 in term 0.
I20260618 22:29:58.059116 24567 raft_consensus.cc:504] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3da7f7dd91441b9aecc72bb24f18d95" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39635 } } peers { permanent_uuid: "5bb209b212104e4bb55097033bc47606" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37359 } } peers { permanent_uuid: "ae65410218aa462ca85d19dc8dc779e9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 41063 } }
I20260618 22:29:58.059702 24567 leader_election.cc:288] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [CANDIDATE]: Term 1 election: Requested vote from peers c3da7f7dd91441b9aecc72bb24f18d95 (127.10.242.195:39635), 5bb209b212104e4bb55097033bc47606 (127.10.242.194:37359)
I20260618 22:29:58.060190 24492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "835fde719ad8498ab26bee5cf44bafe8" candidate_uuid: "ae65410218aa462ca85d19dc8dc779e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3da7f7dd91441b9aecc72bb24f18d95"
I20260618 22:29:58.060281 24358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "835fde719ad8498ab26bee5cf44bafe8" candidate_uuid: "ae65410218aa462ca85d19dc8dc779e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bb209b212104e4bb55097033bc47606"
I20260618 22:29:58.060385 24492 raft_consensus.cc:3037] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:29:58.060495 24358 raft_consensus.cc:3037] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:29:58.062631 24492 raft_consensus.cc:2454] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae65410218aa462ca85d19dc8dc779e9 in term 1.
I20260618 22:29:58.062989 24148 leader_election.cc:302] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae65410218aa462ca85d19dc8dc779e9, c3da7f7dd91441b9aecc72bb24f18d95; no voters:
I20260618 22:29:58.063249 24567 raft_consensus.cc:2781] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:29:58.063512 24358 raft_consensus.cc:2454] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae65410218aa462ca85d19dc8dc779e9 in term 1.
I20260618 22:29:58.063906 24567 raft_consensus.cc:686] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [term 1 LEADER]: Becoming Leader. State: Replica: ae65410218aa462ca85d19dc8dc779e9, State: Running, Role: LEADER
I20260618 22:29:58.064388 24567 consensus_queue.cc:237] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3da7f7dd91441b9aecc72bb24f18d95" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39635 } } peers { permanent_uuid: "5bb209b212104e4bb55097033bc47606" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37359 } } peers { permanent_uuid: "ae65410218aa462ca85d19dc8dc779e9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 41063 } }
I20260618 22:29:58.069242 24060 catalog_manager.cc:5495] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae65410218aa462ca85d19dc8dc779e9 (127.10.242.193). New cstate: current_term: 1 leader_uuid: "ae65410218aa462ca85d19dc8dc779e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3da7f7dd91441b9aecc72bb24f18d95" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bb209b212104e4bb55097033bc47606" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae65410218aa462ca85d19dc8dc779e9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 41063 } health_report { overall_health: HEALTHY } } }
I20260618 22:29:58.103549 24559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:58.097165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39546 (local address 127.10.242.193:41063)
0618 22:29:58.097372 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:58.097376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:58.097455 (+ 79us) server_negotiation.cc:407] Connection header received
0618 22:29:58.097642 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:58.097648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:58.097746 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:58.097904 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:29:58.098343 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:58.099487 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:58.100890 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:58.101509 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:58.101632 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:58.102140 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:29:58.102854 (+ 714us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:29:58.102995 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:58.103152 (+ 157us) server_negotiation.cc:299] Negotiation successful
0618 22:29:58.103239 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:29:58.112744 24492 raft_consensus.cc:1261] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95 [term 1 FOLLOWER]: Refusing update from remote peer ae65410218aa462ca85d19dc8dc779e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:29:58.112823 24358 raft_consensus.cc:1261] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606 [term 1 FOLLOWER]: Refusing update from remote peer ae65410218aa462ca85d19dc8dc779e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:29:58.113775 24567 consensus_queue.cc:1035] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3da7f7dd91441b9aecc72bb24f18d95" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 39635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:29:58.114142 24578 consensus_queue.cc:1035] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bb209b212104e4bb55097033bc47606" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 37359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 22:29:58.120525 24404 tablet.cc:2369] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:29:58.132081 24581 mvcc.cc:205] Tried to move back new op lower bound from 7298342085059747840 to 7298342084875350016. Current Snapshot: MvccSnapshot[applied={T|T < 7298342085059747840}]
I20260618 22:29:58.156534 24060 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:29:58.156867 24060 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:29:58.160302 24060 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fabb7980629643faaa413a42d1b5d1f2: Sending DeleteTablet for 3 replicas of tablet 835fde719ad8498ab26bee5cf44bafe8
I20260618 22:29:58.161556 24472 tablet_service.cc:1514] Processing DeleteTablet for tablet 835fde719ad8498ab26bee5cf44bafe8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:29:58 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:43678
I20260618 22:29:58.161546 24194 tablet_service.cc:1514] Processing DeleteTablet for tablet 835fde719ad8498ab26bee5cf44bafe8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:29:58 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:39530
I20260618 22:29:58.161779 24338 tablet_service.cc:1514] Processing DeleteTablet for tablet 835fde719ad8498ab26bee5cf44bafe8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:29:58 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:40080
I20260618 22:29:58.163405 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 24098
I20260618 22:29:58.182570 24591 tablet_replica.cc:330] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95: stopping tablet replica
I20260618 22:29:58.183113 24591 raft_consensus.cc:2229] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:29:58.183656 24591 raft_consensus.cc:2258] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:29:58.184430 24592 tablet_replica.cc:330] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9: stopping tablet replica
I20260618 22:29:58.184885 24592 raft_consensus.cc:2229] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:29:58.185680 24592 raft_consensus.cc:2258] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:29:58.187045 24593 tablet_replica.cc:330] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606: stopping tablet replica
I20260618 22:29:58.187373 24593 raft_consensus.cc:2229] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:29:58.187773 24593 raft_consensus.cc:2258] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:29:58.187964 24591 ts_tablet_manager.cc:1905] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:29:58.190006 24593 ts_tablet_manager.cc:1905] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:29:58.194583 24592 ts_tablet_manager.cc:1905] T 835fde719ad8498ab26bee5cf44bafe8 P ae65410218aa462ca85d19dc8dc779e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:29:58.196012 24559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:58.164777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39554 (local address 127.10.242.193:41063)
0618 22:29:58.164960 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:58.164967 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:58.175339 (+ 10372us) server_negotiation.cc:407] Connection header received
0618 22:29:58.175832 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:58.175839 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:58.175930 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:58.176018 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:58.178175 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:58.179524 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:58.181605 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:58.182327 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:58.182423 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:58.188481 (+ 6058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:58.188495 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:58.188512 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:58.188563 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:58.192289 (+ 3726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:58.193237 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:58.193246 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:58.193250 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:58.193454 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:58.193949 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:58.193953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:58.193956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:58.194431 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:58.194715 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:58.195323 (+ 608us) server_negotiation.cc:299] Negotiation successful
0618 22:29:58.195581 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:29:58.197301 24593 ts_tablet_manager.cc:1918] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:29:58.197544 24254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:29:58.197530 24593 log.cc:1196] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/835fde719ad8498ab26bee5cf44bafe8
I20260618 22:29:58.198175 24593 ts_tablet_manager.cc:1939] T 835fde719ad8498ab26bee5cf44bafe8 P 5bb209b212104e4bb55097033bc47606: Deleting consensus metadata
I20260618 22:29:58.200104 24591 ts_tablet_manager.cc:1918] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:29:58.200253 24591 log.cc:1196] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/835fde719ad8498ab26bee5cf44bafe8
I20260618 22:29:58.200717 24591 ts_tablet_manager.cc:1939] T 835fde719ad8498ab26bee5cf44bafe8 P c3da7f7dd91441b9aecc72bb24f18d95: Deleting consensus metadata
I20260618 22:29:58.200788 24047 catalog_manager.cc:4841] TS 5bb209b212104e4bb55097033bc47606 (127.10.242.194:37359): tablet 835fde719ad8498ab26bee5cf44bafe8 (table test-table [id=d7e855275be34935a8ed7e620c35188c]) successfully deleted
I20260618 22:29:58.202498 24047 catalog_manager.cc:4841] TS c3da7f7dd91441b9aecc72bb24f18d95 (127.10.242.195:39635): tablet 835fde719ad8498ab26bee5cf44bafe8 (table test-table [id=d7e855275be34935a8ed7e620c35188c]) successfully deleted
I20260618 22:29:58.301570 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 24098
W20260618 22:29:58.315270 24047 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.242.193:41063 (error 108)
I20260618 22:29:58.315408 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 24263
I20260618 22:29:58.316778 24558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:58.316399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:41063 (local address 127.0.0.1:39556)
0618 22:29:58.316555 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:58.316648 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:41063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260618 22:29:58.317013 24047 catalog_manager.cc:4584] TS ae65410218aa462ca85d19dc8dc779e9 (127.10.242.193:41063): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 835fde719ad8498ab26bee5cf44bafe8: Network error: Client connection negotiation failed: client connection to 127.10.242.193:41063: connect: Connection refused (error 111)
I20260618 22:29:58.328181 24556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:58.316156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40094 (local address 127.10.242.194:37359)
0618 22:29:58.316623 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:58.316627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:58.316641 (+ 14us) server_negotiation.cc:407] Connection header received
0618 22:29:58.316690 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:58.316693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:58.316756 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:58.316838 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:58.318182 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:58.318886 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:58.320113 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:58.320478 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:58.320590 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:58.322973 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:58.322987 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:58.323003 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:58.323034 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:58.325646 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:58.326275 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:58.326281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:58.326284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:58.326408 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:58.326763 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:58.326768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:58.326770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:58.327098 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:58.327347 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:58.327667 (+ 320us) server_negotiation.cc:299] Negotiation successful
0618 22:29:58.327862 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365}
I20260618 22:29:58.328773 24398 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:29:58.370560 24558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:58.370107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:41063 (local address 127.0.0.1:39566)
0618 22:29:58.370299 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:58.370403 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:41063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260618 22:29:58.371539 24558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:58.371200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:41063 (local address 127.0.0.1:39570)
0618 22:29:58.371344 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:58.371412 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:41063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260618 22:29:58.417187 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 24263
I20260618 22:29:58.430399 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 24407
I20260618 22:29:58.432228 24558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:58.431702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:41063 (local address 127.0.0.1:39572)
0618 22:29:58.431930 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:58.432023 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:41063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260618 22:29:58.433274 24558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:58.432875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:41063 (local address 127.0.0.1:39576)
0618 22:29:58.433041 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:58.433143 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:41063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260618 22:29:58.443110 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:58.431172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43684 (local address 127.10.242.195:39635)
0618 22:29:58.431326 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0618 22:29:58.431331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:29:58.431413 (+ 82us) server_negotiation.cc:407] Connection header received
0618 22:29:58.431574 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:29:58.431580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:29:58.431675 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:29:58.431779 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:29:58.433343 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:58.434103 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:58.435350 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:29:58.435715 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:29:58.435796 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:29:58.438163 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:29:58.438173 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:29:58.438187 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:29:58.438215 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:29:58.440793 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:58.441436 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:58.441442 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:58.441444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:58.441581 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:29:58.441864 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:29:58.441867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:29:58.441869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:29:58.442201 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:29:58.442415 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0618 22:29:58.442717 (+ 302us) server_negotiation.cc:299] Negotiation successful
0618 22:29:58.442875 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260618 22:29:58.443758 24533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:29:58.513886 24558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:58.513421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:41063 (local address 127.0.0.1:39578)
0618 22:29:58.513626 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:58.513717 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:41063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260618 22:29:58.514832 24558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:29:58.514482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:41063 (local address 127.0.0.1:39580)
0618 22:29:58.514660 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0618 22:29:58.514726 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:41063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260618 22:29:58.533141 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 24407
I20260618 22:29:58.546196 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 24028
I20260618 22:29:58.547478 24091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:29:58.659051 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 24028
I20260618 22:29:58.680462 11211 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260618 22:29:59.015532 11211 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260618 22:29:59.015725 11211 mini_ranger.cc:66] Stopping Ranger...
I20260618 22:29:59.350852 11211 mini_ranger.cc:68] Stopped Ranger
2026-06-18 22:29:59.351 UTC [23080] LOG: received smart shutdown request
2026-06-18 22:29:59.353 UTC [23080] LOG: background worker "logical replication launcher" (PID 23089) exited with exit code 1
2026-06-18 22:29:59.353 UTC [23084] LOG: shutting down
2026-06-18 22:29:59.500 UTC [23080] LOG: database system is shut down
2026-06-18T22:29:59Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52352 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:29:59 dist-test-slave-0956 krb5kdc[24608](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:29:59 dist-test-slave-0956 krb5kdc[24608](info): set up 2 sockets
Jun 18 22:29:59 dist-test-slave-0956 krb5kdc[24608](info): commencing operation
krb5kdc: starting...
W20260618 22:30:00.626928 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:30:00 dist-test-slave-0956 krb5kdc[24608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:30:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:30:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:30:00.860791 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:40539
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:40539,127.10.242.253:36047,127.10.242.252:37887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:30:01.181172 24621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:01.181500 24621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:01.181625 24621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:01.189054 24621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:30:01.189204 24621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:01.189277 24621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:01.189352 24621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:30:01.189397 24621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:30:01.199528 24621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:40539,127.10.242.253:36047,127.10.242.252:37887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:40539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:01.200886 24621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:01.202251 24621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:01.214442 24627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:01.214470 24629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:01.214506 24626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:01.215431 24621 server_base.cc:1034] running on GCE node
I20260618 22:30:01.216251 24621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:01.217785 24621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:01.219007 24621 hybrid_clock.cc:648] HybridClock initialized: now 1781821801218951 us; error 46 us; skew 500 ppm
I20260618 22:30:01.219386 24621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:01 dist-test-slave-0956 krb5kdc[24608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821801, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:01.227016 24621 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:30:01.228648 24621 webserver.cc:460] Webserver started at http://127.10.242.254:44983/ using document root <none> and password file <none>
I20260618 22:30:01.229405 24621 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:01.229530 24621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:01.229828 24621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:30:01.232901 24621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "00a17ddec74d44cda938b54afe21f747"
format_stamp: "Formatted at 2026-06-18 22:30:01 on dist-test-slave-0956"
server_key: "a1122a0d55af6e67aee05cd061804280"
server_key_iv: "c4f10e7a1d6101e3ba8556f3fda76ef0"
server_key_version: "encryptionkey@0"
I20260618 22:30:01.233795 24621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "00a17ddec74d44cda938b54afe21f747"
format_stamp: "Formatted at 2026-06-18 22:30:01 on dist-test-slave-0956"
server_key: "a1122a0d55af6e67aee05cd061804280"
server_key_iv: "c4f10e7a1d6101e3ba8556f3fda76ef0"
server_key_version: "encryptionkey@0"
I20260618 22:30:01.239246 24621 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:30:01.242795 24636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:01.244223 24621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:30:01.244395 24621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "00a17ddec74d44cda938b54afe21f747"
format_stamp: "Formatted at 2026-06-18 22:30:01 on dist-test-slave-0956"
server_key: "a1122a0d55af6e67aee05cd061804280"
server_key_iv: "c4f10e7a1d6101e3ba8556f3fda76ef0"
server_key_version: "encryptionkey@0"
I20260618 22:30:01.244611 24621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:01.260861 24621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:01.265130 24621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:01.265412 24621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:01.283466 24621 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:40539
I20260618 22:30:01.283507 24687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:40539 every 8 connection(s)
I20260618 22:30:01.284870 24621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:30:01.288628 24688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:30:01.291724 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 24621
I20260618 22:30:01.291985 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:30:01.292642 11211 external_mini_cluster.cc:1428] Setting key 8b3800277f85444d84ca76fa4baa68aa
I20260618 22:30:01.295732 24688 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.242.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.253@KRBTEST.COM" created.
Jun 18 22:30:01 dist-test-slave-0956 krb5kdc[24608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821801, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:30:01.323307 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.300667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34608 (local address 127.10.242.254:40539)
0618 22:30:01.301410 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:01.301419 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:01.301474 (+ 55us) server_negotiation.cc:407] Connection header received
0618 22:30:01.302846 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:01.302854 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:01.303218 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:01.303460 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:01.304900 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:01.306656 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:01.308363 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:01.309312 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:01.309524 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:01.315776 (+ 6252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:01.315785 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:01.315799 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:01.315835 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:01.318810 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:01.319564 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:01.319571 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:01.319573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:01.319725 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:01.320100 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:01.320105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:01.320107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:01.320643 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:01.320835 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:01.321462 (+ 627us) server_negotiation.cc:299] Negotiation successful
0618 22:30:01.321804 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":528,"thread_start_us":325,"threads_started":1}
I20260618 22:30:01.323336 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.299267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40539 (local address 127.0.0.1:34608)
0618 22:30:01.300148 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.300198 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:01.302413 (+ 2215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:01.303527 (+ 1114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:01.303533 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:01.304311 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:01.304680 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:01.304700 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:01.306831 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:01.306835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:01.308202 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:01.308216 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:01.309578 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:01.309584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:01.309750 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:01.310407 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:01.310432 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:01.315519 (+ 5087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:01.318971 (+ 3452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:01.318984 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:01.318999 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:01.319435 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:01.319843 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:01.319848 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:01.319851 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:01.320001 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:01.320788 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:01.320796 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:01.321043 (+ 247us) client_negotiation.cc:769] Sending connection context
0618 22:30:01.321324 (+ 281us) client_negotiation.cc:240] Negotiation successful
0618 22:30:01.321943 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":129,"thread_start_us":355,"threads_started":1}
I20260618 22:30:01.326884 24688 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:30:01.327968 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.327544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:36047 (local address 127.0.0.1:39860)
0618 22:30:01.327720 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.327837 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:36047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260618 22:30:01.328208 24638 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.242.253:36047: connect: Connection refused (error 111)
I20260618 22:30:01.329948 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.329547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:36047 (local address 127.0.0.1:39872)
0618 22:30:01.329737 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.329807 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:36047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260618 22:30:01.330163 24688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.253:36047: Network error: Client connection negotiation failed: client connection to 127.10.242.253:36047: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.253.keytab.
Entry for principal kudu/127.10.242.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.253.keytab.
I20260618 22:30:01.341024 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.253:36047
--webserver_interface=127.10.242.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.253.keytab
--principal=kudu/127.10.242.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:40539,127.10.242.253:36047,127.10.242.252:37887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:30:01.379499 24688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } attempt: 1
I20260618 22:30:01.380627 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.380176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:36047 (local address 127.0.0.1:39884)
0618 22:30:01.380368 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.380485 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:36047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260618 22:30:01.381709 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.381361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:36047 (local address 127.0.0.1:39890)
0618 22:30:01.381494 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.381574 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:36047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260618 22:30:01.381927 24688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.253:36047: Network error: Client connection negotiation failed: client connection to 127.10.242.253:36047: connect: Connection refused (error 111)
I20260618 22:30:01.450188 24688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } attempt: 2
I20260618 22:30:01.451396 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.450888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:36047 (local address 127.0.0.1:39902)
0618 22:30:01.451159 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.451252 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:36047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260618 22:30:01.452519 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.452120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:36047 (local address 127.0.0.1:39906)
0618 22:30:01.452303 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.452381 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:36047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260618 22:30:01.452709 24688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.253:36047: Network error: Client connection negotiation failed: client connection to 127.10.242.253:36047: connect: Connection refused (error 111)
I20260618 22:30:01.543965 24688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } attempt: 3
I20260618 22:30:01.545169 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.544671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:36047 (local address 127.0.0.1:39910)
0618 22:30:01.544875 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.544979 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:36047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260618 22:30:01.546379 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.545950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:36047 (local address 127.0.0.1:39912)
0618 22:30:01.546126 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.546212 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:36047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260618 22:30:01.546598 24688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.253:36047: Network error: Client connection negotiation failed: client connection to 127.10.242.253:36047: connect: Connection refused (error 111)
W20260618 22:30:01.661789 24695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:01.662081 24695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:01.662175 24695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:01.669339 24695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:30:01.669458 24695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:01.669512 24695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:01.669566 24695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:30:01.669615 24695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:30:01.679482 24695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:40539,127.10.242.253:36047,127.10.242.252:37887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.253.keytab
--principal=kudu/127.10.242.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.253:36047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.242.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:01.680792 24695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:01.682047 24695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:30:01.689843 24688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } attempt: 4
I20260618 22:30:01.691028 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.690491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:36047 (local address 127.0.0.1:39916)
0618 22:30:01.690806 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.690894 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:36047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260618 22:30:01.692180 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.691682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:36047 (local address 127.0.0.1:39932)
0618 22:30:01.691864 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.691994 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:36047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260618 22:30:01.692575 24688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.253:36047: Network error: Client connection negotiation failed: client connection to 127.10.242.253:36047: connect: Connection refused (error 111)
W20260618 22:30:01.693858 24700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:01.694064 24703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:01.694093 24701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:01.695384 24695 server_base.cc:1034] running on GCE node
I20260618 22:30:01.696218 24695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:01.697860 24695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:01.699077 24695 hybrid_clock.cc:648] HybridClock initialized: now 1781821801699025 us; error 51 us; skew 500 ppm
I20260618 22:30:01.699436 24695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:01 dist-test-slave-0956 krb5kdc[24608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821801, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:01.702634 24695 init.cc:377] Logged in from keytab as kudu/127.10.242.253@KRBTEST.COM (short username kudu)
I20260618 22:30:01.704229 24695 webserver.cc:460] Webserver started at http://127.10.242.253:39067/ using document root <none> and password file <none>
I20260618 22:30:01.704947 24695 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:01.705065 24695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:01.705379 24695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:30:01.708451 24695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-1/data/instance:
uuid: "db23199c6bce4bdaaa2d16aca59e4b41"
format_stamp: "Formatted at 2026-06-18 22:30:01 on dist-test-slave-0956"
server_key: "e91c2858a104fb0d9cff5754d124dc36"
server_key_iv: "cc5a72d86cb4777131ecba4ef18ed304"
server_key_version: "encryptionkey@0"
I20260618 22:30:01.709286 24695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-1/wal/instance:
uuid: "db23199c6bce4bdaaa2d16aca59e4b41"
format_stamp: "Formatted at 2026-06-18 22:30:01 on dist-test-slave-0956"
server_key: "e91c2858a104fb0d9cff5754d124dc36"
server_key_iv: "cc5a72d86cb4777131ecba4ef18ed304"
server_key_version: "encryptionkey@0"
I20260618 22:30:01.714634 24695 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260618 22:30:01.718515 24710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:01.719990 24695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:30:01.720168 24695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-1/wal
uuid: "db23199c6bce4bdaaa2d16aca59e4b41"
format_stamp: "Formatted at 2026-06-18 22:30:01 on dist-test-slave-0956"
server_key: "e91c2858a104fb0d9cff5754d124dc36"
server_key_iv: "cc5a72d86cb4777131ecba4ef18ed304"
server_key_version: "encryptionkey@0"
I20260618 22:30:01.720348 24695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:01.732810 24695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:01.737125 24695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:01.737375 24695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:01.753958 24695 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.253:36047
I20260618 22:30:01.753986 24761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.253:36047 every 8 connection(s)
I20260618 22:30:01.755373 24695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-1/data/info.pb
I20260618 22:30:01.759141 24762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:30:01.761598 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 24695
I20260618 22:30:01.761780 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-1/wal/instance
I20260618 22:30:01.762277 11211 external_mini_cluster.cc:1428] Setting key c33602728b2ed127b6d57d7efb0ef61c
I20260618 22:30:01.766048 24762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.242.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.252@KRBTEST.COM" created.
Jun 18 22:30:01 dist-test-slave-0956 krb5kdc[24608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821801, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.253@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:30:01.790614 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.768932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34620 (local address 127.10.242.254:40539)
0618 22:30:01.769165 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:01.769170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:01.770657 (+ 1487us) server_negotiation.cc:407] Connection header received
0618 22:30:01.772706 (+ 2049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:01.772712 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:01.772794 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:01.772905 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:01.774625 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:01.775672 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:01.777704 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:01.778347 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:01.778499 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:01.784518 (+ 6019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:01.784523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:01.784526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:01.784588 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:01.787262 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:01.788228 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:01.788232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:01.788235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:01.788359 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:01.788780 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:01.788783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:01.788785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:01.789071 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:01.789218 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:01.790026 (+ 808us) server_negotiation.cc:299] Negotiation successful
0618 22:30:01.790277 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:30:01.791375 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.769583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40539 (local address 127.0.0.1:34620)
0618 22:30:01.770456 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.770505 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:01.772416 (+ 1911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:01.773228 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:01.773236 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:01.773803 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:01.774419 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:01.774439 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:01.775880 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:01.775887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:01.777519 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:01.777536 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:01.778605 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:01.778610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:01.778819 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:01.779528 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:01.779552 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:01.784210 (+ 4658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:01.787433 (+ 3223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:01.787444 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:01.787464 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:01.788096 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:01.788453 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:01.788458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:01.788462 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:01.788689 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:01.789228 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:01.789237 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:01.789643 (+ 406us) client_negotiation.cc:769] Sending connection context
0618 22:30:01.789878 (+ 235us) client_negotiation.cc:240] Negotiation successful
0618 22:30:01.790279 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":366,"threads_started":1}
I20260618 22:30:01.793193 24762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:30:01 dist-test-slave-0956 krb5kdc[24608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821801, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.253@KRBTEST.COM for kudu/127.10.242.253@KRBTEST.COM
I20260618 22:30:01.808357 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.793906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:36047 (local address 127.0.0.1:39934)
0618 22:30:01.794076 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.794108 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:01.794198 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:01.795340 (+ 1142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:01.795344 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:01.795782 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:01.795913 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:01.795921 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:01.797228 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:01.797233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:01.798067 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:01.798082 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:01.799374 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:01.799378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:01.799555 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:01.800128 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:01.800146 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:01.802368 (+ 2222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:01.805944 (+ 3576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:01.805948 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:01.805951 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:01.806382 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:01.806800 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:01.806805 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:01.806807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:01.806962 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:01.807547 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:01.807551 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:01.807710 (+ 159us) client_negotiation.cc:769] Sending connection context
0618 22:30:01.807825 (+ 115us) client_negotiation.cc:240] Negotiation successful
0618 22:30:01.808042 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260618 22:30:01.808512 24767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.794160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.10.242.253:36047)
0618 22:30:01.794829 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:01.794836 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:01.794869 (+ 33us) server_negotiation.cc:407] Connection header received
0618 22:30:01.794923 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:01.794929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:01.795174 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:01.795346 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:01.796067 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:01.797102 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:01.798229 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:01.799202 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:01.799348 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:01.802622 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:01.802638 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:01.802654 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:01.802687 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:01.805769 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:01.806514 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:01.806544 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:01.806546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:01.806673 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:01.807055 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:01.807060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:01.807062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:01.807426 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:01.807631 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:01.807961 (+ 330us) server_negotiation.cc:299] Negotiation successful
0618 22:30:01.808161 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":372,"threads_started":1}
I20260618 22:30:01.811282 24762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.252.keytab.
Entry for principal kudu/127.10.242.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.252.keytab.
I20260618 22:30:01.812696 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.812122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38692)
0618 22:30:01.812322 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.812470 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260618 22:30:01.812979 24713 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
I20260618 22:30:01.813633 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.252:37887
--webserver_interface=127.10.242.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.252.keytab
--principal=kudu/127.10.242.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:40539,127.10.242.253:36047,127.10.242.252:37887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:30:01.814734 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.814322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38694)
0618 22:30:01.814513 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.814619 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260618 22:30:01.814971 24762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:37887: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
I20260618 22:30:01.864322 24762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } attempt: 1
I20260618 22:30:01.865651 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.865118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38704)
0618 22:30:01.865331 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.865459 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260618 22:30:01.866797 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.866392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38708)
0618 22:30:01.866559 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.866651 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260618 22:30:01.867081 24762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:37887: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
I20260618 22:30:01.935346 24762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } attempt: 2
I20260618 22:30:01.936550 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.935965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38712)
0618 22:30:01.936245 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.936361 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260618 22:30:01.937610 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.937227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38720)
0618 22:30:01.937374 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.937463 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260618 22:30:01.937865 24762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:37887: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
I20260618 22:30:01.991881 24688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } attempt: 5
Jun 18 22:30:01 dist-test-slave-0956 krb5kdc[24608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821801, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for kudu/127.10.242.253@KRBTEST.COM
I20260618 22:30:02.005568 24767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.992806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39950 (local address 127.10.242.253:36047)
0618 22:30:01.993022 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:01.993028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:01.993050 (+ 22us) server_negotiation.cc:407] Connection header received
0618 22:30:01.993160 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:01.993165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:01.993248 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:01.993330 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:01.994201 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:01.994878 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:01.996036 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:01.996485 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:01.996599 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:02.000037 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:02.000042 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:02.000046 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:02.000085 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:02.002765 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.003504 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.003508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.003511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.003647 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.004067 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.004070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.004072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.004382 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:02.004534 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:02.004935 (+ 401us) server_negotiation.cc:299] Negotiation successful
0618 22:30:02.005215 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260618 22:30:02.005566 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:01.992686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:36047 (local address 127.0.0.1:39950)
0618 22:30:01.992874 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:01.992895 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:01.993023 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:01.993392 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:01.993397 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:01.993821 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:01.994038 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:01.994050 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:01.995019 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:01.995023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:01.995888 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:01.995901 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:01.996634 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:01.996639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:01.996810 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:01.997449 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:01.997468 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:01.999811 (+ 2343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:02.002951 (+ 3140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:02.002957 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:02.002962 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:02.003363 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:02.003764 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:02.003770 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:02.003774 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:02.003945 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:02.004508 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:02.004513 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:02.004680 (+ 167us) client_negotiation.cc:769] Sending connection context
0618 22:30:02.004806 (+ 126us) client_negotiation.cc:240] Negotiation successful
0618 22:30:02.005211 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260618 22:30:02.006700 24688 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:30:02.008083 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.007599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38728)
0618 22:30:02.007807 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:02.007901 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260618 22:30:02.009335 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.008894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38732)
0618 22:30:02.009060 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:02.009208 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260618 22:30:02.009549 24688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:37887: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
I20260618 22:30:02.029146 24762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } attempt: 3
I20260618 22:30:02.030429 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.029848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38744)
0618 22:30:02.030098 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:02.030222 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260618 22:30:02.031701 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.031215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38756)
0618 22:30:02.031455 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:02.031552 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260618 22:30:02.031955 24762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:37887: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
I20260618 22:30:02.060814 24688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } attempt: 1
I20260618 22:30:02.062062 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.061563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38758)
0618 22:30:02.061776 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:02.061891 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260618 22:30:02.063033 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.062712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38760)
0618 22:30:02.062841 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:02.062911 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260618 22:30:02.063242 24688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:37887: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
I20260618 22:30:02.131449 24688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } attempt: 2
I20260618 22:30:02.132733 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.132184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38768)
0618 22:30:02.132412 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:02.132557 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260618 22:30:02.133973 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.133536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38778)
0618 22:30:02.133728 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:02.133828 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260618 22:30:02.134212 24688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:37887: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
W20260618 22:30:02.140748 24769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:02.141067 24769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:02.141222 24769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:02.148814 24769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:30:02.148933 24769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:02.148998 24769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:02.149052 24769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:30:02.149132 24769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:30:02.159471 24769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:40539,127.10.242.253:36047,127.10.242.252:37887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.252.keytab
--principal=kudu/127.10.242.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.252:37887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.242.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:02.160770 24769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:02.162066 24769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:02.173405 24774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:02.173396 24775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:02.173521 24777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:02.174144 24769 server_base.cc:1034] running on GCE node
I20260618 22:30:02.174885 24769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:02.175160 24762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } attempt: 4
I20260618 22:30:02.176276 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.175760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38786)
0618 22:30:02.175995 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:02.176090 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260618 22:30:02.176637 24769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:02.177294 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.176912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38790)
0618 22:30:02.177062 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:02.177171 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260618 22:30:02.177541 24762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:37887: Network error: Client connection negotiation failed: client connection to 127.10.242.252:37887: connect: Connection refused (error 111)
I20260618 22:30:02.177829 24769 hybrid_clock.cc:648] HybridClock initialized: now 1781821802177772 us; error 51 us; skew 500 ppm
I20260618 22:30:02.178159 24769 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:02 dist-test-slave-0956 krb5kdc[24608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821802, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:02.181176 24769 init.cc:377] Logged in from keytab as kudu/127.10.242.252@KRBTEST.COM (short username kudu)
I20260618 22:30:02.182564 24769 webserver.cc:460] Webserver started at http://127.10.242.252:39969/ using document root <none> and password file <none>
I20260618 22:30:02.183297 24769 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:02.183422 24769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:02.183718 24769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:30:02.186741 24769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-2/data/instance:
uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4"
format_stamp: "Formatted at 2026-06-18 22:30:02 on dist-test-slave-0956"
server_key: "54ea3b9ec80986d92d61a59c81df988c"
server_key_iv: "76ec5471eee9d877ff5b4a46103183ba"
server_key_version: "encryptionkey@0"
I20260618 22:30:02.187507 24769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-2/wal/instance:
uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4"
format_stamp: "Formatted at 2026-06-18 22:30:02 on dist-test-slave-0956"
server_key: "54ea3b9ec80986d92d61a59c81df988c"
server_key_iv: "76ec5471eee9d877ff5b4a46103183ba"
server_key_version: "encryptionkey@0"
I20260618 22:30:02.192493 24769 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260618 22:30:02.195860 24784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:02.197234 24769 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260618 22:30:02.197389 24769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-2/wal
uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4"
format_stamp: "Formatted at 2026-06-18 22:30:02 on dist-test-slave-0956"
server_key: "54ea3b9ec80986d92d61a59c81df988c"
server_key_iv: "76ec5471eee9d877ff5b4a46103183ba"
server_key_version: "encryptionkey@0"
I20260618 22:30:02.197563 24769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:02.215229 24769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:02.219084 24769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:02.219354 24769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:02.235893 24769 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.252:37887
I20260618 22:30:02.235916 24835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.252:37887 every 8 connection(s)
I20260618 22:30:02.237287 24769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-2/data/info.pb
I20260618 22:30:02.240410 24688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } attempt: 3
I20260618 22:30:02.240773 24836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:30:02.244545 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 24769
I20260618 22:30:02.244735 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821604371818-11211-0/minicluster-data/master-2/wal/instance
I20260618 22:30:02.245304 11211 external_mini_cluster.cc:1428] Setting key 7ec011b4e223acf3074b8fb6abf5b2a6
I20260618 22:30:02.252002 24836 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:30:02 dist-test-slave-0956 krb5kdc[24608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821801, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for kudu/127.10.242.252@KRBTEST.COM
I20260618 22:30:02.268245 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.241115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38798)
0618 22:30:02.241399 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:02.241419 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:02.241532 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:02.246797 (+ 5265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:02.246804 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:02.247333 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:02.247573 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:02.247587 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.252232 (+ 4645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.252238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:02.253819 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:02.253839 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.255355 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.255362 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:02.255573 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:02.256281 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:02.256307 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:02.259395 (+ 3088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:02.265231 (+ 5836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:02.265236 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:02.265240 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:02.265645 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:02.266076 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:02.266079 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:02.266081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:02.266212 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:02.267217 (+ 1005us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:02.267222 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:02.267419 (+ 197us) client_negotiation.cc:769] Sending connection context
0618 22:30:02.267584 (+ 165us) client_negotiation.cc:240] Negotiation successful
0618 22:30:02.267867 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160}
I20260618 22:30:02.269873 24838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.242020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38798 (local address 127.10.242.252:37887)
0618 22:30:02.242783 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:02.242798 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:02.242847 (+ 49us) server_negotiation.cc:407] Connection header received
0618 22:30:02.245710 (+ 2863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:02.245740 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:02.246296 (+ 556us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:02.247844 (+ 1548us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:02.248573 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.252000 (+ 3427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.254044 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.255157 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.259726 (+ 4569us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:02.260181 (+ 455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:02.260192 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:02.260216 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:02.260262 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:02.265040 (+ 4778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.265778 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.265784 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.265791 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.265962 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.266319 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.266322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.266324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.267050 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:02.267226 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:02.267870 (+ 644us) server_negotiation.cc:299] Negotiation successful
0618 22:30:02.268318 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":324,"threads_started":1}
Jun 18 22:30:02 dist-test-slave-0956 krb5kdc[24608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821802, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.252@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
Jun 18 22:30:02 dist-test-slave-0956 krb5kdc[24608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.252@KRBTEST.COM
I20260618 22:30:02.277262 24688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747: Bootstrap starting.
I20260618 22:30:02.277715 24840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.246370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38810 (local address 127.10.242.252:37887)
0618 22:30:02.261659 (+ 15289us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:02.261665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:02.261701 (+ 36us) server_negotiation.cc:407] Connection header received
0618 22:30:02.261752 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:02.261756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:02.261845 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:02.261941 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:02.263574 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.264562 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.266482 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.267207 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.267303 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:02.272542 (+ 5239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:02.272547 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:02.272550 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:02.272579 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:02.274957 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.275787 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.275790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.275793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.275906 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.276275 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.276278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.276280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.276548 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:02.276807 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:02.277133 (+ 326us) server_negotiation.cc:299] Negotiation successful
0618 22:30:02.277342 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":14809,"thread_start_us":14958,"threads_started":1}
I20260618 22:30:02.278095 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.255872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34626 (local address 127.10.242.254:40539)
0618 22:30:02.256068 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:02.256074 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:02.262299 (+ 6225us) server_negotiation.cc:407] Connection header received
0618 22:30:02.262605 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:02.262612 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:02.262687 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:02.262750 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:02.264394 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.265174 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.266981 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.267603 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.267699 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:02.271316 (+ 3617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:02.271323 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:02.271328 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:02.271367 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:02.274940 (+ 3573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.275757 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.275763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.275766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.275931 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.276367 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.276372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.276375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.276725 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:02.276848 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:02.277541 (+ 693us) server_negotiation.cc:299] Negotiation successful
0618 22:30:02.277756 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260618 22:30:02.279206 24841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.255365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40539 (local address 127.0.0.1:34626)
0618 22:30:02.262152 (+ 6787us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:02.262203 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:02.262429 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:02.262967 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:02.262975 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:02.263811 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:02.264235 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:02.264249 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.265325 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.265330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:02.266847 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:02.266858 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.267935 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.267941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:02.268235 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:02.268816 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:02.268842 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:02.271087 (+ 2245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:02.275072 (+ 3985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:02.275084 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:02.275091 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:02.275633 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:02.276063 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:02.276068 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:02.276071 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:02.276239 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:02.276959 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:02.276967 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:02.277205 (+ 238us) client_negotiation.cc:769] Sending connection context
0618 22:30:02.278489 (+ 1284us) client_negotiation.cc:240] Negotiation successful
0618 22:30:02.278812 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6491,"mutex_wait_us":2,"thread_start_us":6783,"threads_started":1}
I20260618 22:30:02.280843 24836 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:30:02.281330 24688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747: Neither blocks nor log segments found. Creating new log.
I20260618 22:30:02.282697 24688 log.cc:824] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:30:02 dist-test-slave-0956 krb5kdc[24608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
Jun 18 22:30:02 dist-test-slave-0956 krb5kdc[24608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821802, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.252@KRBTEST.COM for kudu/127.10.242.253@KRBTEST.COM
I20260618 22:30:02.296643 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.279252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34628 (local address 127.10.242.254:40539)
0618 22:30:02.279406 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:02.279411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:02.279495 (+ 84us) server_negotiation.cc:407] Connection header received
0618 22:30:02.279655 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:02.279661 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:02.279719 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:02.279797 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:02.281136 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.282145 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.283851 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.284509 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.284607 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:02.289704 (+ 5097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:02.289710 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:02.289714 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:02.289754 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:02.293247 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.294169 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.294175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.294180 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.294346 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.294801 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.294806 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.294809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.295232 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:02.295484 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:02.296026 (+ 542us) server_negotiation.cc:299] Negotiation successful
0618 22:30:02.296317 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:30:02.297379 24841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.281477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:36047 (local address 127.0.0.1:39966)
0618 22:30:02.281843 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:02.281860 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:02.281949 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:02.282283 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:02.282288 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:02.282722 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:02.282895 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:02.282907 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.284372 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.284378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:02.285855 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:02.285870 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.287105 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.287112 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:02.287372 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:02.287956 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:02.287980 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:02.290621 (+ 2641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:02.294573 (+ 3952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:02.294579 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:02.294583 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:02.295167 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:02.295633 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:02.295638 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:02.295642 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:02.295808 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:02.296463 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:02.296466 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:02.296585 (+ 119us) client_negotiation.cc:769] Sending connection context
0618 22:30:02.296706 (+ 121us) client_negotiation.cc:240] Negotiation successful
0618 22:30:02.296978 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246}
I20260618 22:30:02.297410 24767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.281834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39966 (local address 127.10.242.253:36047)
0618 22:30:02.282009 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:02.282014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:02.282031 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:30:02.282085 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:02.282090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:02.282159 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:02.282245 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:02.283068 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.284157 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.286035 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.286746 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.286842 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:02.290864 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:02.290871 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:02.290877 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:02.290923 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:02.294388 (+ 3465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.295314 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.295320 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.295324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.295488 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.295929 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.295935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.295939 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.296282 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:02.296388 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:02.296826 (+ 438us) server_negotiation.cc:299] Negotiation successful
0618 22:30:02.297029 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260618 22:30:02.298689 24836 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:30:02.299100 24688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747: No bootstrap required, opened a new log
Jun 18 22:30:02 dist-test-slave-0956 krb5kdc[24608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821802, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.252@KRBTEST.COM for kudu/127.10.242.252@KRBTEST.COM
Jun 18 22:30:02 dist-test-slave-0956 krb5kdc[24608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.253@KRBTEST.COM
I20260618 22:30:02.308820 24688 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } }
I20260618 22:30:02.309271 24688 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:30:02.309396 24688 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00a17ddec74d44cda938b54afe21f747, State: Initialized, Role: FOLLOWER
I20260618 22:30:02.310111 24688 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } }
I20260618 22:30:02.311940 24843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } } }
I20260618 22:30:02.312436 24843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:30:02.313411 24688 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:30:02.314332 24841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.299459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38826)
0618 22:30:02.299681 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:02.299698 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:02.300047 (+ 349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:02.300391 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:02.300396 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:02.300883 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:02.301112 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:02.301125 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.302223 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.302228 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:02.303303 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:02.303318 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.304256 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.304262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:02.304450 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:02.305351 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:02.305388 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:02.308138 (+ 2750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:02.311460 (+ 3322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:02.311465 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:02.311469 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:02.311955 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:02.312353 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:02.312357 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:02.312360 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:02.312499 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:02.313268 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:02.313274 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:02.313440 (+ 166us) client_negotiation.cc:769] Sending connection context
0618 22:30:02.313607 (+ 167us) client_negotiation.cc:240] Negotiation successful
0618 22:30:02.313939 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":4}
I20260618 22:30:02.314360 24840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.299788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38826 (local address 127.10.242.252:37887)
0618 22:30:02.299968 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:02.299974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:02.299991 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:30:02.300182 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:02.300187 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:02.300254 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:02.300350 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:02.301238 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.302109 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.303449 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.304084 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.304200 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:02.308296 (+ 4096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:02.308302 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:02.308307 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:02.308346 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:02.311304 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.312085 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.312090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.312093 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.312228 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.312610 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.312614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.312617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.313013 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:02.313213 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:02.313713 (+ 500us) server_negotiation.cc:299] Negotiation successful
0618 22:30:02.314075 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":161,"server-negotiator.queue_time_us":64}
I20260618 22:30:02.316763 24767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.299384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39976 (local address 127.10.242.253:36047)
0618 22:30:02.299538 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:02.299543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:02.299559 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:30:02.299723 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:02.299728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:02.299796 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:02.299877 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:02.301386 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.302612 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.304087 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.304682 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.304792 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:02.310242 (+ 5450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:02.310249 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:02.310254 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:02.310297 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:02.313404 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.314332 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.314338 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.314342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.314501 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.314964 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.314968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.314971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.315326 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:02.315626 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:02.316200 (+ 574us) server_negotiation.cc:299] Negotiation successful
0618 22:30:02.316431 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260618 22:30:02.319340 24836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4: Bootstrap starting.
I20260618 22:30:02.324697 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:30:02.324712 24836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4: Neither blocks nor log segments found. Creating new log.
I20260618 22:30:02.325984 24836 log.cc:824] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4: Log is configured to *not* fsync() on all Append() calls
W20260618 22:30:02.329252 24854 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:30:02.329439 24854 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:30:02.327494 24621 master_runner.cc:186] Error getting master registration for 127.10.242.253:36047: OK, instance_id { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" instance_seqno: 1781821801749157 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 22:30:02.340008 24621 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260618 22:30:02.342815 24836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4: No bootstrap required, opened a new log
I20260618 22:30:02.355432 24836 raft_consensus.cc:348] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } }
I20260618 22:30:02.356055 24836 raft_consensus.cc:374] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:30:02.356226 24836 raft_consensus.cc:729] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 473f6a3b9fa44fbbb76656d0fd2c6ba4, State: Initialized, Role: FOLLOWER
I20260618 22:30:02.357141 24836 consensus_queue.cc:260] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } }
I20260618 22:30:02.364203 24863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } } }
I20260618 22:30:02.366657 24863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:30:02.366534 24836 sys_catalog.cc:564] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:30:02.368263 24840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.343104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38836 (local address 127.10.242.252:37887)
0618 22:30:02.343277 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:02.343284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:02.350101 (+ 6817us) server_negotiation.cc:407] Connection header received
0618 22:30:02.350285 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:02.350290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:02.350392 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:02.350505 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:02.352697 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.353859 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.355963 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.356600 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.356700 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:02.360623 (+ 3923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:02.360631 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:02.360636 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:02.360683 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:02.364464 (+ 3781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.365325 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.365330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.365334 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.365494 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.365985 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.365990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.365993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.366355 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:02.366592 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:02.367172 (+ 580us) server_negotiation.cc:299] Negotiation successful
0618 22:30:02.367456 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:30:02.368558 24843 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:30:02.368863 24843 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } }
I20260618 22:30:02.370556 24843 leader_election.cc:288] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db23199c6bce4bdaaa2d16aca59e4b41 (127.10.242.253:36047), 473f6a3b9fa44fbbb76656d0fd2c6ba4 (127.10.242.252:37887)
I20260618 22:30:02.371889 24811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "00a17ddec74d44cda938b54afe21f747" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" is_pre_election: true
I20260618 22:30:02.372913 24811 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00a17ddec74d44cda938b54afe21f747 in term 0.
I20260618 22:30:02.373606 24639 leader_election.cc:302] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00a17ddec74d44cda938b54afe21f747, 473f6a3b9fa44fbbb76656d0fd2c6ba4; no voters:
I20260618 22:30:02.374202 24843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:30:02.374368 24843 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:30:02.374485 24843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:30:02.378262 24843 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } }
I20260618 22:30:02.380307 24811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "00a17ddec74d44cda938b54afe21f747" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4"
I20260618 22:30:02.380534 24811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:30:02.383760 24811 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00a17ddec74d44cda938b54afe21f747 in term 1.
I20260618 22:30:02.384387 24639 leader_election.cc:302] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00a17ddec74d44cda938b54afe21f747, 473f6a3b9fa44fbbb76656d0fd2c6ba4; no voters:
I20260618 22:30:02.387377 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.342743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34640 (local address 127.10.242.254:40539)
0618 22:30:02.342938 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:02.342945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:02.361631 (+ 18686us) server_negotiation.cc:407] Connection header received
0618 22:30:02.361900 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:02.361906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:02.362003 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:02.362080 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:02.364085 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.365287 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.367582 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.368224 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.368316 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:02.372110 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:02.372119 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:02.372124 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:02.372175 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:02.375893 (+ 3718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.376802 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.376808 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.376812 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.377012 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.385214 (+ 8202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.385221 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.385226 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.385672 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:02.385950 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:02.386649 (+ 699us) server_negotiation.cc:299] Negotiation successful
0618 22:30:02.386963 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260618 22:30:02.387631 24843 leader_election.cc:288] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [CANDIDATE]: Term 1 election: Requested vote from peers db23199c6bce4bdaaa2d16aca59e4b41 (127.10.242.253:36047), 473f6a3b9fa44fbbb76656d0fd2c6ba4 (127.10.242.252:37887)
I20260618 22:30:02.388531 24843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:30:02.389155 24843 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [term 1 LEADER]: Becoming Leader. State: Replica: 00a17ddec74d44cda938b54afe21f747, State: Running, Role: LEADER
I20260618 22:30:02.390003 24863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:30:02.390187 24767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.361854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39990 (local address 127.10.242.253:36047)
0618 22:30:02.362036 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:02.362041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:02.368648 (+ 6607us) server_negotiation.cc:407] Connection header received
0618 22:30:02.368948 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:02.368954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:02.369036 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:02.369124 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:02.371117 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.372161 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.376045 (+ 3884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.376739 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.376833 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:02.382347 (+ 5514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:02.382354 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:02.382359 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:02.382404 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:02.385887 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.387300 (+ 1413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.387306 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.387311 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.387494 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.388174 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.388179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.388182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.388574 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:02.388808 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:02.389626 (+ 818us) server_negotiation.cc:299] Negotiation successful
0618 22:30:02.389866 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260618 22:30:02.390210 24863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } }
I20260618 22:30:02.391903 24863 leader_election.cc:288] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 00a17ddec74d44cda938b54afe21f747 (127.10.242.254:40539), db23199c6bce4bdaaa2d16aca59e4b41 (127.10.242.253:36047)
I20260618 22:30:02.392635 24843 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } }
I20260618 22:30:02.395424 24663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00a17ddec74d44cda938b54afe21f747" is_pre_election: true
I20260618 22:30:02.399691 24865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00a17ddec74d44cda938b54afe21f747. Latest consensus state: current_term: 1 leader_uuid: "00a17ddec74d44cda938b54afe21f747" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } } }
I20260618 22:30:02.400286 24865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [sys.catalog]: This master's current role is: LEADER
I20260618 22:30:02.402258 24868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:30:02.406057 24868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:30:02.421599 24811 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [term 1 FOLLOWER]: Refusing update from remote peer 00a17ddec74d44cda938b54afe21f747: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:30:02.422539 24843 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [LEADER]: Connected to new peer: Peer: permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:30:02.436226 24863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00a17ddec74d44cda938b54afe21f747. Latest consensus state: current_term: 1 leader_uuid: "00a17ddec74d44cda938b54afe21f747" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } } }
I20260618 22:30:02.436620 24863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:30:02.443831 24870 mvcc.cc:205] Tried to move back new op lower bound from 7298342102703017984 to 7298342102605053952. Current Snapshot: MvccSnapshot[applied={T|T < 7298342102703017984}]
I20260618 22:30:02.452811 24863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00a17ddec74d44cda938b54afe21f747" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } } }
I20260618 22:30:02.453158 24863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:30:02.460762 24865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00a17ddec74d44cda938b54afe21f747" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } } }
I20260618 22:30:02.461103 24865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [sys.catalog]: This master's current role is: LEADER
I20260618 22:30:02.463428 24868 catalog_manager.cc:1347] Generated new cluster ID: 2f40d211d3044caa8ea42f6d5bc1aec1
I20260618 22:30:02.463527 24868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:30:02.463845 24883 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260618 22:30:02.476744 24762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } attempt: 5
Jun 18 22:30:02 dist-test-slave-0956 krb5kdc[24608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821801, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.253@KRBTEST.COM for kudu/127.10.242.252@KRBTEST.COM
I20260618 22:30:02.472985 24769 master_runner.cc:186] Error getting master registration for 127.10.242.253:36047: OK, instance_id { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" instance_seqno: 1781821801749157 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 22:30:02.495525 24769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260618 22:30:02.497565 24868 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:30:02.499549 24868 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:30:02.500317 24840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.477963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38852 (local address 127.10.242.252:37887)
0618 22:30:02.478135 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:02.478140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:02.478159 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:30:02.478215 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:02.478220 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:02.478291 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:02.478383 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:02.479483 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.480587 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.482402 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.483082 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.483203 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:02.493039 (+ 9836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:02.493046 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:02.493051 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:02.493108 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:02.496747 (+ 3639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.497696 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.497703 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.497707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.497952 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:02.498480 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:02.498485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:02.498489 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:02.498845 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:02.498987 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:02.499776 (+ 789us) server_negotiation.cc:299] Negotiation successful
0618 22:30:02.499991 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260618 22:30:02.500981 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:02.477294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:37887 (local address 127.0.0.1:38852)
0618 22:30:02.477639 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:02.477659 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:02.477765 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:02.478557 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:02.478562 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:02.479101 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:02.479308 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:02.479321 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.480745 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.480751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:02.482231 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:02.482247 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:02.483414 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:02.483421 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:02.483640 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:02.484399 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:02.484424 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:02.492808 (+ 8384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:02.496899 (+ 4091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:02.496925 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:02.496930 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:02.497503 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:02.498079 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:02.498118 (+ 39us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:02.498122 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:02.498354 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:02.499046 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:02.499052 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:02.499205 (+ 153us) client_negotiation.cc:769] Sending connection context
0618 22:30:02.499331 (+ 126us) client_negotiation.cc:240] Negotiation successful
0618 22:30:02.500604 (+ 1273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218}
I20260618 22:30:02.508044 24762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41: Bootstrap starting.
I20260618 22:30:02.512043 24762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41: Neither blocks nor log segments found. Creating new log.
I20260618 22:30:02.513433 24762 log.cc:824] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41: Log is configured to *not* fsync() on all Append() calls
I20260618 22:30:02.523329 24762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41: No bootstrap required, opened a new log
I20260618 22:30:02.531826 24868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747: Generated new TSK 0
I20260618 22:30:02.534916 24762 raft_consensus.cc:348] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } }
I20260618 22:30:02.535374 24762 raft_consensus.cc:374] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:30:02.535531 24762 raft_consensus.cc:729] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db23199c6bce4bdaaa2d16aca59e4b41, State: Initialized, Role: FOLLOWER
I20260618 22:30:02.536211 24762 consensus_queue.cc:260] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } }
I20260618 22:30:02.536809 24868 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:30:02.537689 24885 sys_catalog.cc:455] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } } }
I20260618 22:30:02.538482 24885 sys_catalog.cc:458] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:30:02.538839 24762 sys_catalog.cc:564] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:30:02.538877 24737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "00a17ddec74d44cda938b54afe21f747" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" is_pre_election: true
I20260618 22:30:02.538916 24735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" is_pre_election: true
I20260618 22:30:02.539490 24735 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 473f6a3b9fa44fbbb76656d0fd2c6ba4 in term 0.
I20260618 22:30:02.539772 24736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "00a17ddec74d44cda938b54afe21f747" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db23199c6bce4bdaaa2d16aca59e4b41"
I20260618 22:30:02.540058 24736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:30:02.540122 24786 leader_election.cc:302] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 473f6a3b9fa44fbbb76656d0fd2c6ba4, db23199c6bce4bdaaa2d16aca59e4b41; no voters: 00a17ddec74d44cda938b54afe21f747
I20260618 22:30:02.540781 24863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260618 22:30:02.540894 24863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00a17ddec74d44cda938b54afe21f747)
I20260618 22:30:02.540989 24863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:30:02.543745 24736 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00a17ddec74d44cda938b54afe21f747 in term 1.
I20260618 22:30:02.544019 24863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } }
I20260618 22:30:02.544598 24734 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [term 1 FOLLOWER]: Refusing update from remote peer 00a17ddec74d44cda938b54afe21f747: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:30:02.545185 24663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00a17ddec74d44cda938b54afe21f747"
I20260618 22:30:02.546463 24865 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:30:02.546903 24863 leader_election.cc:288] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [CANDIDATE]: Term 2 election: Requested vote from peers 00a17ddec74d44cda938b54afe21f747 (127.10.242.254:40539), db23199c6bce4bdaaa2d16aca59e4b41 (127.10.242.253:36047)
I20260618 22:30:02.547643 24734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db23199c6bce4bdaaa2d16aca59e4b41"
I20260618 22:30:02.547928 24734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:30:02.551065 24734 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 473f6a3b9fa44fbbb76656d0fd2c6ba4 in term 2.
I20260618 22:30:02.551645 24786 leader_election.cc:302] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 473f6a3b9fa44fbbb76656d0fd2c6ba4, db23199c6bce4bdaaa2d16aca59e4b41; no voters: 00a17ddec74d44cda938b54afe21f747
I20260618 22:30:02.552145 24863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [term 2 FOLLOWER]: Leader election won for term 2
I20260618 22:30:02.552919 24736 raft_consensus.cc:1226] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [term 2 FOLLOWER]: Rejecting Update request from peer 00a17ddec74d44cda938b54afe21f747 for earlier term 1. Current term is 2. Ops: [1.1-1.4]
I20260618 22:30:02.554754 24865 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:30:02.555130 24865 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [term 1 LEADER]: Stepping down as leader of term 1
I20260618 22:30:02.555235 24865 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00a17ddec74d44cda938b54afe21f747, State: Running, Role: LEADER
I20260618 22:30:02.555598 24865 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } }
I20260618 22:30:02.556203 24865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:30:02.559309 24863 raft_consensus.cc:686] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [term 2 LEADER]: Becoming Leader. State: Replica: 473f6a3b9fa44fbbb76656d0fd2c6ba4, State: Running, Role: LEADER
I20260618 22:30:02.559983 24863 consensus_queue.cc:237] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } }
I20260618 22:30:02.562266 24887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 473f6a3b9fa44fbbb76656d0fd2c6ba4. Latest consensus state: current_term: 2 leader_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } } }
I20260618 22:30:02.562621 24887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [sys.catalog]: This master's current role is: LEADER
I20260618 22:30:02.563529 24893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:30:02.576030 24893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:30:02.577811 24893 catalog_manager.cc:1259] Loaded cluster ID: 2f40d211d3044caa8ea42f6d5bc1aec1
I20260618 22:30:02.577986 24893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:30:02.582259 24893 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:30:02.583976 24893 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4: Loaded TSK: 0
I20260618 22:30:02.585911 24893 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260618 22:30:02.603371 24899 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:30:02.603570 24899 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260618 22:30:02.740057 24716 tablet.cc:2369] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:30:03.049351 24736 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [term 2 FOLLOWER]: Refusing update from remote peer 473f6a3b9fa44fbbb76656d0fd2c6ba4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260618 22:30:03.050076 24887 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:30:03.057132 24906 mvcc.cc:205] Tried to move back new op lower bound from 7298342102703017984 to 7298342102605053952. Current Snapshot: MvccSnapshot[applied={T|T < 7298342102703017984}]
I20260618 22:30:03.059298 24905 sys_catalog.cc:455] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 473f6a3b9fa44fbbb76656d0fd2c6ba4. Latest consensus state: current_term: 2 leader_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } } }
I20260618 22:30:03.059660 24905 sys_catalog.cc:458] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:30:03.059942 24908 sys_catalog.cc:455] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } } }
I20260618 22:30:03.060295 24908 sys_catalog.cc:458] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:30:03.061538 24863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } } }
I20260618 22:30:03.061863 24863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [sys.catalog]: This master's current role is: LEADER
I20260618 22:30:03.062213 24908 sys_catalog.cc:455] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } } }
I20260618 22:30:03.062412 24908 sys_catalog.cc:458] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:30:03.062506 24663 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [term 2 FOLLOWER]: Refusing update from remote peer 473f6a3b9fa44fbbb76656d0fd2c6ba4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260618 22:30:03.063256 24863 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260618 22:30:03.065701 24915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } } }
I20260618 22:30:03.065980 24915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:30:03.065855 24913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 473f6a3b9fa44fbbb76656d0fd2c6ba4. Latest consensus state: current_term: 2 leader_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } } }
I20260618 22:30:03.066103 24913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:30:03.331759 24854 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 00a17ddec74d44cda938b54afe21f747: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 22:30:03.573938 24863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00a17ddec74d44cda938b54afe21f747" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40539 } } peers { permanent_uuid: "db23199c6bce4bdaaa2d16aca59e4b41" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 36047 } } peers { permanent_uuid: "473f6a3b9fa44fbbb76656d0fd2c6ba4" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 37887 } } }
I20260618 22:30:03.574261 24863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473f6a3b9fa44fbbb76656d0fd2c6ba4 [sys.catalog]: This master's current role is: LEADER
I20260618 22:30:03.605630 24899 catalog_manager.cc:1259] Loaded cluster ID: 2f40d211d3044caa8ea42f6d5bc1aec1
I20260618 22:30:03.605743 24899 catalog_manager.cc:1552] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41: loading cluster ID for follower catalog manager: success
I20260618 22:30:03.608256 24899 catalog_manager.cc:1574] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41: acquiring CA information for follower catalog manager: success
I20260618 22:30:03.610208 24899 catalog_manager.cc:1602] T 00000000000000000000000000000000 P db23199c6bce4bdaaa2d16aca59e4b41: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 22:30:03.704206 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 24621
I20260618 22:30:03.705533 24683 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:03.816460 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 24621
I20260618 22:30:03.833694 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 24695
I20260618 22:30:03.834812 24757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:03.942039 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 24695
I20260618 22:30:03.958315 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 24769
I20260618 22:30:03.959596 24831 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:04.066850 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 24769
2026-06-18T22:30:04Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4461 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:30:04 dist-test-slave-0956 krb5kdc[24925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:30:04 dist-test-slave-0956 krb5kdc[24925](info): set up 2 sockets
Jun 18 22:30:04 dist-test-slave-0956 krb5kdc[24925](info): commencing operation
krb5kdc: starting...
W20260618 22:30:05.093374 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:30:05 dist-test-slave-0956 krb5kdc[24925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:30:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:30:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:30:05.310864 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:40465
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:45023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:40465
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:30:05.638024 24938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:05.638337 24938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:05.638469 24938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:05.645797 24938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:30:05.645912 24938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:05.645987 24938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:05.646056 24938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:30:05.646113 24938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:30:05.656090 24938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:45023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:40465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:40465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:05.657478 24938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:05.658787 24938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:05.670316 24943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:05.670365 24944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:05.670348 24946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:05.670964 24938 server_base.cc:1034] running on GCE node
I20260618 22:30:05.671744 24938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:05.673249 24938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:05.674472 24938 hybrid_clock.cc:648] HybridClock initialized: now 1781821805674413 us; error 45 us; skew 500 ppm
I20260618 22:30:05.674816 24938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:05 dist-test-slave-0956 krb5kdc[24925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821805, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:05.677872 24938 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:30:05.679375 24938 webserver.cc:460] Webserver started at http://127.10.242.254:41645/ using document root <none> and password file <none>
I20260618 22:30:05.680018 24938 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:05.680114 24938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:05.680375 24938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:30:05.683337 24938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "2076df1bf0364621ac3e9082dd40bdbf"
format_stamp: "Formatted at 2026-06-18 22:30:05 on dist-test-slave-0956"
server_key: "ed3a70e44d7f4c5add0a359bbcf0ffef"
server_key_iv: "71924fcd0294b3e6310897fdc24581cd"
server_key_version: "encryptionkey@0"
I20260618 22:30:05.684023 24938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "2076df1bf0364621ac3e9082dd40bdbf"
format_stamp: "Formatted at 2026-06-18 22:30:05 on dist-test-slave-0956"
server_key: "ed3a70e44d7f4c5add0a359bbcf0ffef"
server_key_iv: "71924fcd0294b3e6310897fdc24581cd"
server_key_version: "encryptionkey@0"
I20260618 22:30:05.689109 24938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260618 22:30:05.692690 24953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:05.694182 24938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260618 22:30:05.694375 24938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "2076df1bf0364621ac3e9082dd40bdbf"
format_stamp: "Formatted at 2026-06-18 22:30:05 on dist-test-slave-0956"
server_key: "ed3a70e44d7f4c5add0a359bbcf0ffef"
server_key_iv: "71924fcd0294b3e6310897fdc24581cd"
server_key_version: "encryptionkey@0"
I20260618 22:30:05.694586 24938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:05.714509 24938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:05.718839 24938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:05.719112 24938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:05.736732 24938 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:40465
I20260618 22:30:05.736734 25004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:40465 every 8 connection(s)
I20260618 22:30:05.738152 24938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:30:05.741922 25005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:30:05.743741 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 24938
I20260618 22:30:05.743959 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:30:05.744732 11211 external_mini_cluster.cc:1428] Setting key c7105ace67556670f7201fb196dad5c5
I20260618 22:30:05.756028 25005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf: Bootstrap starting.
Jun 18 22:30:05 dist-test-slave-0956 krb5kdc[24925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:30:05.759928 25005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf: Neither blocks nor log segments found. Creating new log.
I20260618 22:30:05.761199 25005 log.cc:824] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf: Log is configured to *not* fsync() on all Append() calls
I20260618 22:30:05.765206 25005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf: No bootstrap required, opened a new log
I20260618 22:30:05.769192 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:05.746636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42582 (local address 127.10.242.254:40465)
0618 22:30:05.747408 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:05.747420 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:05.747467 (+ 47us) server_negotiation.cc:407] Connection header received
0618 22:30:05.749017 (+ 1550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:05.749044 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:05.749496 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:05.749813 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:05.751687 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:05.753036 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:05.755005 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:05.755774 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:05.755920 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:05.760873 (+ 4953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:05.760885 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:05.760900 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:05.760946 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:05.764463 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:05.765293 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:05.765302 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:05.765310 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:05.765519 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:05.765897 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:05.765903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:05.765906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:05.766611 (+ 705us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:05.766907 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:05.767286 (+ 379us) server_negotiation.cc:299] Negotiation successful
0618 22:30:05.767786 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":339,"threads_started":1}
I20260618 22:30:05.773299 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:30:05.774580 25005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2076df1bf0364621ac3e9082dd40bdbf" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40465 } }
I20260618 22:30:05.774953 25005 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:30:05.775051 25005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2076df1bf0364621ac3e9082dd40bdbf, State: Initialized, Role: FOLLOWER
I20260618 22:30:05.775653 25005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2076df1bf0364621ac3e9082dd40bdbf" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40465 } }
I20260618 22:30:05.775897 25005 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:30:05.776031 25005 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:30:05.776176 25005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:30:05.778378 25005 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2076df1bf0364621ac3e9082dd40bdbf" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40465 } }
I20260618 22:30:05.778877 25005 leader_election.cc:302] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2076df1bf0364621ac3e9082dd40bdbf; no voters:
I20260618 22:30:05.779429 25005 leader_election.cc:288] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:30:05.779558 25010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:30:05.780207 25010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf [term 1 LEADER]: Becoming Leader. State: Replica: 2076df1bf0364621ac3e9082dd40bdbf, State: Running, Role: LEADER
I20260618 22:30:05.780747 25010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2076df1bf0364621ac3e9082dd40bdbf" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40465 } }
I20260618 22:30:05.781533 25005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:30:05.783532 25012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2076df1bf0364621ac3e9082dd40bdbf. Latest consensus state: current_term: 1 leader_uuid: "2076df1bf0364621ac3e9082dd40bdbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2076df1bf0364621ac3e9082dd40bdbf" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40465 } } }
I20260618 22:30:05.783509 25011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2076df1bf0364621ac3e9082dd40bdbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2076df1bf0364621ac3e9082dd40bdbf" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40465 } } }
I20260618 22:30:05.784016 25012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf [sys.catalog]: This master's current role is: LEADER
I20260618 22:30:05.784016 25011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf [sys.catalog]: This master's current role is: LEADER
I20260618 22:30:05.784900 25016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:30:05.789510 25016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:30:05.799018 25016 catalog_manager.cc:1347] Generated new cluster ID: ea74977c4cc5405ca2a88900e884c0d4
I20260618 22:30:05.799172 25016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:30:05.813421 25016 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:30:05.814760 25016 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:30:05.828720 25016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2076df1bf0364621ac3e9082dd40bdbf: Generated new TSK 0
I20260618 22:30:05.829895 25016 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:30:05.884999 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 24938
I20260618 22:30:05.886258 25000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:05.982791 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 24938
2026-06-18T22:30:05Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1911 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:30:06 dist-test-slave-0956 krb5kdc[25033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:30:06 dist-test-slave-0956 krb5kdc[25033](info): set up 2 sockets
Jun 18 22:30:06 dist-test-slave-0956 krb5kdc[25033](info): commencing operation
krb5kdc: starting...
W20260618 22:30:07.009358 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.001s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:30:07 dist-test-slave-0956 krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:30:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:30:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:30:07.234972 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:35805
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:35805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:30:07.559454 25046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:07.559757 25046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:07.559866 25046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:07.567211 25046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:30:07.567335 25046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:07.567410 25046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:07.567489 25046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:30:07.567548 25046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:30:07.577601 25046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:35805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:35805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:07.578898 25046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:07.580164 25046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:07.591512 25051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:07.591485 25054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:07.591455 25052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:07.592502 25046 server_base.cc:1034] running on GCE node
I20260618 22:30:07.593341 25046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:07.594800 25046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:07.596045 25046 hybrid_clock.cc:648] HybridClock initialized: now 1781821807595976 us; error 50 us; skew 500 ppm
I20260618 22:30:07.596549 25046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:07 dist-test-slave-0956 krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821807, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:07.603318 25046 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:30:07.604940 25046 webserver.cc:460] Webserver started at http://127.10.242.254:44395/ using document root <none> and password file <none>
I20260618 22:30:07.605741 25046 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:07.605857 25046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:07.606189 25046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:30:07.609417 25046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "bffd12ba538c464b868b911063445d7e"
format_stamp: "Formatted at 2026-06-18 22:30:07 on dist-test-slave-0956"
server_key: "d21f3092844ba29bbac61e0c99f86b05"
server_key_iv: "299a75de26f134e15214cd31c48a2637"
server_key_version: "encryptionkey@0"
I20260618 22:30:07.610226 25046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "bffd12ba538c464b868b911063445d7e"
format_stamp: "Formatted at 2026-06-18 22:30:07 on dist-test-slave-0956"
server_key: "d21f3092844ba29bbac61e0c99f86b05"
server_key_iv: "299a75de26f134e15214cd31c48a2637"
server_key_version: "encryptionkey@0"
I20260618 22:30:07.615931 25046 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260618 22:30:07.619782 25061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:07.621428 25046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260618 22:30:07.621615 25046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "bffd12ba538c464b868b911063445d7e"
format_stamp: "Formatted at 2026-06-18 22:30:07 on dist-test-slave-0956"
server_key: "d21f3092844ba29bbac61e0c99f86b05"
server_key_iv: "299a75de26f134e15214cd31c48a2637"
server_key_version: "encryptionkey@0"
I20260618 22:30:07.621819 25046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:07.636559 25046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:07.640833 25046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:07.641149 25046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:07.658843 25046 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:35805
I20260618 22:30:07.658875 25112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:35805 every 8 connection(s)
I20260618 22:30:07.660230 25046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:30:07.664058 25113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:30:07.667042 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 25046
I20260618 22:30:07.667268 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:30:07.668108 11211 external_mini_cluster.cc:1428] Setting key f8351ab8ae6188b190ec3426b3d2412f
I20260618 22:30:07.677119 25113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e: Bootstrap starting.
I20260618 22:30:07.681022 25113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e: Neither blocks nor log segments found. Creating new log.
I20260618 22:30:07.682416 25113 log.cc:824] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:30:07 dist-test-slave-0956 krb5kdc[25033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:30:07.685921 25113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e: No bootstrap required, opened a new log
I20260618 22:30:07.693063 25116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:07.670368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38104 (local address 127.10.242.254:35805)
0618 22:30:07.671210 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:07.671223 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:07.671265 (+ 42us) server_negotiation.cc:407] Connection header received
0618 22:30:07.672949 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:07.672975 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:07.673478 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:07.673878 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:07.675694 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:07.677113 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:07.678992 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:07.679803 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:07.679951 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:07.684923 (+ 4972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:07.684932 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:07.684949 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:07.684992 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:07.688425 (+ 3433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:07.689271 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:07.689278 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:07.689285 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:07.689475 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:07.689906 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:07.689910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:07.689912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:07.690560 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:07.690896 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:07.691426 (+ 530us) server_negotiation.cc:299] Negotiation successful
0618 22:30:07.691902 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":406,"threads_started":1}
I20260618 22:30:07.694294 25113 raft_consensus.cc:348] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffd12ba538c464b868b911063445d7e" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35805 } }
I20260618 22:30:07.694685 25113 raft_consensus.cc:374] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:30:07.694777 25113 raft_consensus.cc:729] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bffd12ba538c464b868b911063445d7e, State: Initialized, Role: FOLLOWER
I20260618 22:30:07.695356 25113 consensus_queue.cc:260] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffd12ba538c464b868b911063445d7e" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35805 } }
I20260618 22:30:07.695600 25113 raft_consensus.cc:388] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:30:07.695722 25113 raft_consensus.cc:482] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:30:07.695892 25113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:30:07.698257 25113 raft_consensus.cc:504] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffd12ba538c464b868b911063445d7e" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35805 } }
I20260618 22:30:07.698784 25113 leader_election.cc:302] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bffd12ba538c464b868b911063445d7e; no voters:
I20260618 22:30:07.699399 25113 leader_election.cc:288] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:30:07.699543 25118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:30:07.700629 25118 raft_consensus.cc:686] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 1 LEADER]: Becoming Leader. State: Replica: bffd12ba538c464b868b911063445d7e, State: Running, Role: LEADER
I20260618 22:30:07.701221 25118 consensus_queue.cc:237] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffd12ba538c464b868b911063445d7e" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35805 } }
I20260618 22:30:07.701663 25113 sys_catalog.cc:564] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:30:07.706393 25119 sys_catalog.cc:455] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bffd12ba538c464b868b911063445d7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffd12ba538c464b868b911063445d7e" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35805 } } }
I20260618 22:30:07.706789 25119 sys_catalog.cc:458] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [sys.catalog]: This master's current role is: LEADER
I20260618 22:30:07.707508 25120 sys_catalog.cc:455] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bffd12ba538c464b868b911063445d7e. Latest consensus state: current_term: 1 leader_uuid: "bffd12ba538c464b868b911063445d7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffd12ba538c464b868b911063445d7e" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35805 } } }
I20260618 22:30:07.707742 25126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:30:07.707818 25120 sys_catalog.cc:458] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [sys.catalog]: This master's current role is: LEADER
I20260618 22:30:07.714427 25126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:30:07.724045 25126 catalog_manager.cc:1347] Generated new cluster ID: bf63df4f06ed405ebb7fa01545ceaed8
I20260618 22:30:07.724181 25126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:30:07.736195 25126 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:30:07.737645 25126 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:30:07.758975 25126 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e: Generated new TSK 0
I20260618 22:30:07.759886 25126 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
I20260618 22:30:07.821269 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:35805
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:30:08.147148 25139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:08.147446 25139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:08.147573 25139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:08.155047 25139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:08.155174 25139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:08.155319 25139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:30:08.165153 25139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:35805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:08.166505 25139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:08.167816 25139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:08.180909 25144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:08.180971 25147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:08.180938 25145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:08.181757 25139 server_base.cc:1034] running on GCE node
I20260618 22:30:08.182344 25139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:08.183276 25139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:08.184477 25139 hybrid_clock.cc:648] HybridClock initialized: now 1781821808184414 us; error 56 us; skew 500 ppm
I20260618 22:30:08.184782 25139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:08 dist-test-slave-0956 krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821808, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:08.187925 25139 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:30:08.189459 25139 webserver.cc:460] Webserver started at http://127.10.242.193:39771/ using document root <none> and password file <none>
I20260618 22:30:08.190100 25139 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:08.190197 25139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:08.190451 25139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:30:08.193511 25139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "87cf49fcc4194b5c9429af4a95fc9276"
format_stamp: "Formatted at 2026-06-18 22:30:08 on dist-test-slave-0956"
server_key: "ba3695311fb55c4c288cc1c013c54009"
server_key_iv: "a5576b23f636a677ca4993fc06de6ca5"
server_key_version: "encryptionkey@0"
I20260618 22:30:08.194252 25139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "87cf49fcc4194b5c9429af4a95fc9276"
format_stamp: "Formatted at 2026-06-18 22:30:08 on dist-test-slave-0956"
server_key: "ba3695311fb55c4c288cc1c013c54009"
server_key_iv: "a5576b23f636a677ca4993fc06de6ca5"
server_key_version: "encryptionkey@0"
I20260618 22:30:08.199437 25139 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:30:08.202993 25154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:08.204388 25139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:30:08.204532 25139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "87cf49fcc4194b5c9429af4a95fc9276"
format_stamp: "Formatted at 2026-06-18 22:30:08 on dist-test-slave-0956"
server_key: "ba3695311fb55c4c288cc1c013c54009"
server_key_iv: "a5576b23f636a677ca4993fc06de6ca5"
server_key_version: "encryptionkey@0"
I20260618 22:30:08.204708 25139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:08.218863 25139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:08.222911 25139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:08.223173 25139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:08.224113 25139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:30:08.225481 25139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:30:08.225553 25139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:08.225661 25139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:30:08.225715 25139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:08.255318 25139 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:46011
I20260618 22:30:08.255394 25266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:46011 every 8 connection(s)
I20260618 22:30:08.256561 25139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:30:08.262810 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 25139
I20260618 22:30:08.262993 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:30:08.263598 11211 external_mini_cluster.cc:1428] Setting key 901cbf1b359f766602a6ebea39ef6a23
Jun 18 22:30:08 dist-test-slave-0956 krb5kdc[25033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821808, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:30:08.276111 25269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:08.259178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:45701 (local address 127.10.242.254:35805)
0618 22:30:08.259590 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:08.259594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:08.260530 (+ 936us) server_negotiation.cc:407] Connection header received
0618 22:30:08.262401 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:08.262408 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:08.262483 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:08.262576 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:08.264224 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:08.265065 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:08.266697 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:08.267085 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:08.267196 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:08.270517 (+ 3321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:08.270521 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:08.270525 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:08.270557 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:08.272947 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:08.273794 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:08.273799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:08.273801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:08.273952 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:08.274391 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:08.274395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:08.274397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:08.274705 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:08.274820 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:08.275667 (+ 847us) server_negotiation.cc:299] Negotiation successful
0618 22:30:08.275848 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":189,"threads_started":1}
I20260618 22:30:08.277397 25270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:08.259711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:45701)
0618 22:30:08.260376 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:08.260421 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:08.262087 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:08.262830 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:08.262840 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:08.263424 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:08.264049 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:08.264067 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:08.265236 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:08.265240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:08.266547 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:08.266559 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:08.267298 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:08.267306 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:08.267534 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:08.268162 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:08.268184 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:08.270297 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:08.273107 (+ 2810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:08.273117 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:08.273135 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:08.273665 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:08.274055 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:08.274059 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:08.274063 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:08.274294 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:08.274816 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:08.274825 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:08.275235 (+ 410us) client_negotiation.cc:769] Sending connection context
0618 22:30:08.275514 (+ 279us) client_negotiation.cc:240] Negotiation successful
0618 22:30:08.276054 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":251,"threads_started":1}
I20260618 22:30:08.278949 25267 heartbeater.cc:347] Connected to a master server at 127.10.242.254:35805
I20260618 22:30:08.279276 25267 heartbeater.cc:464] Registering TS with master...
I20260618 22:30:08.280083 25267 heartbeater.cc:511] Master 127.10.242.254:35805 requested a full tablet report, sending...
I20260618 22:30:08.282333 25078 ts_manager.cc:194] Registered new tserver with Master: 87cf49fcc4194b5c9429af4a95fc9276 (127.10.242.193:46011)
I20260618 22:30:08.283782 25078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:45701
WARNING: no policy specified for kudu/127.10.242.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
I20260618 22:30:08.313292 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:35805
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:30:08.625900 25273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:08.626199 25273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:08.626334 25273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:08.633648 25273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:08.633774 25273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:08.633934 25273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:30:08.643608 25273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:35805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:08.644889 25273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:08.646201 25273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:08.658944 25278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:08.658875 25279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:08.658907 25281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:08.662604 25273 server_base.cc:1034] running on GCE node
I20260618 22:30:08.663223 25273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:08.664193 25273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:08.665446 25273 hybrid_clock.cc:648] HybridClock initialized: now 1781821808665387 us; error 51 us; skew 500 ppm
I20260618 22:30:08.665771 25273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:08 dist-test-slave-0956 krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821808, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:08.668927 25273 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:30:08.670413 25273 webserver.cc:460] Webserver started at http://127.10.242.194:43495/ using document root <none> and password file <none>
I20260618 22:30:08.671072 25273 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:08.671182 25273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:08.671466 25273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:30:08.674399 25273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "cc33e2b9b6ed4040bc239bfeae5229a4"
format_stamp: "Formatted at 2026-06-18 22:30:08 on dist-test-slave-0956"
server_key: "39c155f6c1d3099b8e88140b6b5c04a5"
server_key_iv: "c6c2a8e75ba9ba4ba9504114c02fc366"
server_key_version: "encryptionkey@0"
I20260618 22:30:08.675133 25273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "cc33e2b9b6ed4040bc239bfeae5229a4"
format_stamp: "Formatted at 2026-06-18 22:30:08 on dist-test-slave-0956"
server_key: "39c155f6c1d3099b8e88140b6b5c04a5"
server_key_iv: "c6c2a8e75ba9ba4ba9504114c02fc366"
server_key_version: "encryptionkey@0"
I20260618 22:30:08.680460 25273 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260618 22:30:08.684095 25288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:08.685505 25273 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:30:08.685653 25273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "cc33e2b9b6ed4040bc239bfeae5229a4"
format_stamp: "Formatted at 2026-06-18 22:30:08 on dist-test-slave-0956"
server_key: "39c155f6c1d3099b8e88140b6b5c04a5"
server_key_iv: "c6c2a8e75ba9ba4ba9504114c02fc366"
server_key_version: "encryptionkey@0"
I20260618 22:30:08.685901 25273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:08.703249 25273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:08.707108 25273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:08.707371 25273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:08.708334 25273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:30:08.709856 25273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:30:08.709942 25273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:08.710054 25273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:30:08.710104 25273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:08.739833 25273 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:33811
I20260618 22:30:08.739881 25400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:33811 every 8 connection(s)
I20260618 22:30:08.741055 25273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:30:08.744048 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 25273
I20260618 22:30:08.744252 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:30:08.744671 11211 external_mini_cluster.cc:1428] Setting key 13eb7fdcebf923b1a4a23e2141762e8f
Jun 18 22:30:08 dist-test-slave-0956 krb5kdc[25033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821808, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:30:08.761328 25269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:08.743591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:41211 (local address 127.10.242.254:35805)
0618 22:30:08.743815 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:08.743822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:08.745297 (+ 1475us) server_negotiation.cc:407] Connection header received
0618 22:30:08.747115 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:08.747120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:08.747202 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:08.747302 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:08.748736 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:08.749718 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:08.751203 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:08.751752 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:08.751871 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:08.755669 (+ 3798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:08.755674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:08.755677 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:08.755727 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:08.758209 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:08.759032 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:08.759035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:08.759038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:08.759179 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:08.759604 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:08.759607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:08.759610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:08.759877 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:08.760014 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:08.760724 (+ 710us) server_negotiation.cc:299] Negotiation successful
0618 22:30:08.760968 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260618 22:30:08.762291 25403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:08.744301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:41211)
0618 22:30:08.745121 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:08.745168 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:08.746861 (+ 1693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:08.747526 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:08.747534 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:08.748016 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:08.748578 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:08.748595 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:08.749850 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:08.749854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:08.751078 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:08.751089 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:08.751926 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:08.751931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:08.752142 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:08.752758 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:08.752785 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:08.755442 (+ 2657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:08.758362 (+ 2920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:08.758372 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:08.758391 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:08.758938 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:08.759308 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:08.759312 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:08.759314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:08.759497 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:08.760003 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:08.760011 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:08.760356 (+ 345us) client_negotiation.cc:769] Sending connection context
0618 22:30:08.760607 (+ 251us) client_negotiation.cc:240] Negotiation successful
0618 22:30:08.761174 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":301,"threads_started":1}
I20260618 22:30:08.763697 25401 heartbeater.cc:347] Connected to a master server at 127.10.242.254:35805
I20260618 22:30:08.764040 25401 heartbeater.cc:464] Registering TS with master...
I20260618 22:30:08.764951 25401 heartbeater.cc:511] Master 127.10.242.254:35805 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.242.195@KRBTEST.COM; defaulting to no policy
I20260618 22:30:08.766552 25078 ts_manager.cc:194] Registered new tserver with Master: cc33e2b9b6ed4040bc239bfeae5229a4 (127.10.242.194:33811)
I20260618 22:30:08.767350 25078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:41211
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
I20260618 22:30:08.793579 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:35805
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:30:09.108860 25406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:09.109215 25406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:09.109349 25406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:09.116804 25406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:09.116927 25406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:09.117101 25406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:30:09.127132 25406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:35805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:09.128410 25406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:09.129834 25406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:09.143036 25414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:09.143043 25412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:09.143337 25411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:09.143992 25406 server_base.cc:1034] running on GCE node
I20260618 22:30:09.144611 25406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:09.145665 25406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:09.146907 25406 hybrid_clock.cc:648] HybridClock initialized: now 1781821809146848 us; error 51 us; skew 500 ppm
I20260618 22:30:09.147308 25406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:09 dist-test-slave-0956 krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821809, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:09.150727 25406 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:30:09.152259 25406 webserver.cc:460] Webserver started at http://127.10.242.195:38425/ using document root <none> and password file <none>
I20260618 22:30:09.152966 25406 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:09.153103 25406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:09.153417 25406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:30:09.156401 25406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "c84e78806f334957b5ecc9f59882f86b"
format_stamp: "Formatted at 2026-06-18 22:30:09 on dist-test-slave-0956"
server_key: "9c1afa46cc3be1eb2de26465ec16843d"
server_key_iv: "6567709e029397ceb1ed94afa45fbb39"
server_key_version: "encryptionkey@0"
I20260618 22:30:09.157234 25406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "c84e78806f334957b5ecc9f59882f86b"
format_stamp: "Formatted at 2026-06-18 22:30:09 on dist-test-slave-0956"
server_key: "9c1afa46cc3be1eb2de26465ec16843d"
server_key_iv: "6567709e029397ceb1ed94afa45fbb39"
server_key_version: "encryptionkey@0"
I20260618 22:30:09.162448 25406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260618 22:30:09.166023 25421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:09.167472 25406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:30:09.167614 25406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "c84e78806f334957b5ecc9f59882f86b"
format_stamp: "Formatted at 2026-06-18 22:30:09 on dist-test-slave-0956"
server_key: "9c1afa46cc3be1eb2de26465ec16843d"
server_key_iv: "6567709e029397ceb1ed94afa45fbb39"
server_key_version: "encryptionkey@0"
I20260618 22:30:09.167806 25406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:09.186864 25406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:09.191184 25406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:09.191452 25406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:09.192420 25406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:30:09.193832 25406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:30:09.193917 25406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:09.194032 25406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:30:09.194080 25406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:09.223944 25406 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:42763
I20260618 22:30:09.223994 25533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:42763 every 8 connection(s)
I20260618 22:30:09.225327 25406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:30:09.234712 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 25406
I20260618 22:30:09.234905 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:30:09.235363 11211 external_mini_cluster.cc:1428] Setting key b630d06ce611cbc107c84e4fc63cae17
Jun 18 22:30:09 dist-test-slave-0956 krb5kdc[25033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821809, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:30:09.245942 25269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:09.227939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:43087 (local address 127.10.242.254:35805)
0618 22:30:09.228103 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:09.228107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:09.229549 (+ 1442us) server_negotiation.cc:407] Connection header received
0618 22:30:09.231477 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:09.231484 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:09.231576 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:09.231689 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:09.233426 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.234273 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.236057 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.236565 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.236713 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:09.240338 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:09.240342 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:09.240346 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:09.240382 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:09.243039 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:09.243753 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:09.243778 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:09.243781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:09.243905 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:09.244266 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:09.244271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:09.244274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:09.244557 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:09.244671 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:09.245456 (+ 785us) server_negotiation.cc:299] Negotiation successful
0618 22:30:09.245672 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260618 22:30:09.246881 25536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:09.228386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:43087)
0618 22:30:09.229363 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:09.229415 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:09.231163 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:09.231971 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:09.231980 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:09.232557 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:09.233235 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:09.233257 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.234475 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.234479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:09.235900 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:09.235913 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.236807 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.236816 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:09.237142 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:09.237842 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:09.237879 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:09.240126 (+ 2247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:09.243190 (+ 3064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:09.243197 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:09.243211 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:09.243640 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:09.243990 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:09.243993 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:09.243995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:09.244177 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:09.244667 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:09.244673 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:09.245029 (+ 356us) client_negotiation.cc:769] Sending connection context
0618 22:30:09.245311 (+ 282us) client_negotiation.cc:240] Negotiation successful
0618 22:30:09.245729 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":371,"threads_started":1}
I20260618 22:30:09.248384 25534 heartbeater.cc:347] Connected to a master server at 127.10.242.254:35805
I20260618 22:30:09.248653 25534 heartbeater.cc:464] Registering TS with master...
I20260618 22:30:09.249414 25534 heartbeater.cc:511] Master 127.10.242.254:35805 requested a full tablet report, sending...
I20260618 22:30:09.250998 25077 ts_manager.cc:194] Registered new tserver with Master: c84e78806f334957b5ecc9f59882f86b (127.10.242.195:42763)
I20260618 22:30:09.251899 25077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:43087
I20260618 22:30:09.263785 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:30:09.281767 25269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:09.268743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38114 (local address 127.10.242.254:35805)
0618 22:30:09.269007 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:09.269015 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:09.269386 (+ 371us) server_negotiation.cc:407] Connection header received
0618 22:30:09.269584 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:09.269589 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:09.269667 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:09.269767 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:09.271316 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.272140 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.273612 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.274057 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.274200 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:09.276664 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:09.276669 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:09.276672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:09.276710 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:09.279125 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:09.279837 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:09.279841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:09.279843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:09.279976 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:09.280339 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:09.280343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:09.280345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:09.280669 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:09.280909 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:09.281244 (+ 335us) server_negotiation.cc:299] Negotiation successful
0618 22:30:09.281460 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260618 22:30:09.286952 25267 heartbeater.cc:503] Master 127.10.242.254:35805 was elected leader, sending a full tablet report...
I20260618 22:30:09.286134 25077 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:30:09.293679 25077 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:30:09.328022 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:09.305901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:46011 (local address 127.0.0.1:35486)
0618 22:30:09.317824 (+ 11923us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:09.317876 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:09.318117 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:09.319167 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:09.319177 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:09.319211 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:09.319675 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:09.319688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.322089 (+ 2401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.322096 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:09.324893 (+ 2797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:09.324906 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.326948 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.326954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:09.327182 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:09.327239 (+ 57us) client_negotiation.cc:769] Sending connection context
0618 22:30:09.327367 (+ 128us) client_negotiation.cc:240] Negotiation successful
0618 22:30:09.327561 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11644,"mutex_wait_us":102,"thread_start_us":11491,"threads_started":1}
I20260618 22:30:09.328215 25548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:09.306202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35486 (local address 127.10.242.193:46011)
0618 22:30:09.317367 (+ 11165us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:09.317379 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:09.318464 (+ 1085us) server_negotiation.cc:407] Connection header received
0618 22:30:09.318581 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:09.318590 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:09.318897 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:09.319890 (+ 993us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:09.320166 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.321914 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.325049 (+ 3135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.326677 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.326874 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:09.327025 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:09.327740 (+ 715us) server_negotiation.cc:299] Negotiation successful
0618 22:30:09.327917 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10920,"thread_start_us":10711,"threads_started":1}
I20260618 22:30:09.328253 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:09.306345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35366 (local address 127.10.242.194:33811)
0618 22:30:09.317844 (+ 11499us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:09.317853 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:09.318632 (+ 779us) server_negotiation.cc:407] Connection header received
0618 22:30:09.318960 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:09.318968 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:09.319245 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:09.319428 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:09.320405 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.322092 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.325332 (+ 3240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.326683 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.326841 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:09.327025 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:09.327825 (+ 800us) server_negotiation.cc:299] Negotiation successful
0618 22:30:09.327992 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11299,"thread_start_us":11512,"threads_started":1}
I20260618 22:30:09.328707 25547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:09.306198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:33811 (local address 127.0.0.1:35366)
0618 22:30:09.318495 (+ 12297us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:09.318514 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:09.318770 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:09.319740 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:09.319745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:09.319775 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:09.320219 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:09.320231 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.322300 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.322306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:09.325177 (+ 2871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:09.325191 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.327179 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.327186 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:09.327355 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:09.327385 (+ 30us) client_negotiation.cc:769] Sending connection context
0618 22:30:09.327469 (+ 84us) client_negotiation.cc:240] Negotiation successful
0618 22:30:09.327591 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12163,"thread_start_us":12013,"threads_started":1}
I20260618 22:30:09.332254 25202 tablet_service.cc:1467] Processing CreateTablet for tablet c418a2a4d38a46d79099520da714a5ba (DEFAULT_TABLE table=test-table [id=3f40acbe3d2146b3bd07a8901c349101]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:30:09.332993 25550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:09.318699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:42763 (local address 127.0.0.1:57954)
0618 22:30:09.323326 (+ 4627us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:09.323344 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:09.323456 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:09.325397 (+ 1941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:09.325401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:09.325428 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:09.325798 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:09.325807 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.328205 (+ 2398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.328210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:09.330101 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:09.330114 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.332245 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.332252 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:09.332415 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:09.332449 (+ 34us) client_negotiation.cc:769] Sending connection context
0618 22:30:09.332552 (+ 103us) client_negotiation.cc:240] Negotiation successful
0618 22:30:09.332682 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4503,"thread_start_us":4347,"threads_started":1}
I20260618 22:30:09.332435 25336 tablet_service.cc:1467] Processing CreateTablet for tablet c418a2a4d38a46d79099520da714a5ba (DEFAULT_TABLE table=test-table [id=3f40acbe3d2146b3bd07a8901c349101]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:30:09.333071 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:09.318801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57954 (local address 127.10.242.195:42763)
0618 22:30:09.324757 (+ 5956us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:09.324766 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:09.324802 (+ 36us) server_negotiation.cc:407] Connection header received
0618 22:30:09.324893 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:09.324902 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:09.325197 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:09.325421 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:09.325966 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.328045 (+ 2079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.330297 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.332037 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.332228 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:09.332474 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:09.332605 (+ 131us) server_negotiation.cc:299] Negotiation successful
0618 22:30:09.332764 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5772,"thread_start_us":5614,"threads_started":1}
I20260618 22:30:09.333891 25202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c418a2a4d38a46d79099520da714a5ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:30:09.333942 25336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c418a2a4d38a46d79099520da714a5ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:30:09.336925 25469 tablet_service.cc:1467] Processing CreateTablet for tablet c418a2a4d38a46d79099520da714a5ba (DEFAULT_TABLE table=test-table [id=3f40acbe3d2146b3bd07a8901c349101]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:30:09.338405 25469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c418a2a4d38a46d79099520da714a5ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:30:09.346601 25552 tablet_bootstrap.cc:492] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4: Bootstrap starting.
I20260618 22:30:09.348541 25553 tablet_bootstrap.cc:492] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276: Bootstrap starting.
I20260618 22:30:09.350481 25552 tablet_bootstrap.cc:654] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4: Neither blocks nor log segments found. Creating new log.
I20260618 22:30:09.351786 25552 log.cc:824] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4: Log is configured to *not* fsync() on all Append() calls
I20260618 22:30:09.352006 25553 tablet_bootstrap.cc:654] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276: Neither blocks nor log segments found. Creating new log.
I20260618 22:30:09.352103 25554 tablet_bootstrap.cc:492] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b: Bootstrap starting.
I20260618 22:30:09.353385 25553 log.cc:824] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276: Log is configured to *not* fsync() on all Append() calls
I20260618 22:30:09.355679 25554 tablet_bootstrap.cc:654] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b: Neither blocks nor log segments found. Creating new log.
I20260618 22:30:09.356420 25552 tablet_bootstrap.cc:492] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4: No bootstrap required, opened a new log
I20260618 22:30:09.356690 25553 tablet_bootstrap.cc:492] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276: No bootstrap required, opened a new log
I20260618 22:30:09.356930 25552 ts_tablet_manager.cc:1397] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260618 22:30:09.357008 25553 ts_tablet_manager.cc:1397] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260618 22:30:09.357262 25554 log.cc:824] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:30:09.360436 25554 tablet_bootstrap.cc:492] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b: No bootstrap required, opened a new log
I20260618 22:30:09.360739 25554 ts_tablet_manager.cc:1397] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260618 22:30:09.364787 25552 raft_consensus.cc:348] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } }
I20260618 22:30:09.365207 25552 raft_consensus.cc:374] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:30:09.365299 25552 raft_consensus.cc:729] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc33e2b9b6ed4040bc239bfeae5229a4, State: Initialized, Role: FOLLOWER
I20260618 22:30:09.365921 25552 consensus_queue.cc:260] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } }
I20260618 22:30:09.367133 25401 heartbeater.cc:503] Master 127.10.242.254:35805 was elected leader, sending a full tablet report...
I20260618 22:30:09.367626 25552 ts_tablet_manager.cc:1428] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20260618 22:30:09.368256 25553 raft_consensus.cc:348] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } }
I20260618 22:30:09.368870 25553 raft_consensus.cc:374] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:30:09.368968 25553 raft_consensus.cc:729] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87cf49fcc4194b5c9429af4a95fc9276, State: Initialized, Role: FOLLOWER
I20260618 22:30:09.369618 25553 consensus_queue.cc:260] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } }
I20260618 22:30:09.370978 25554 raft_consensus.cc:348] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } }
I20260618 22:30:09.371450 25554 raft_consensus.cc:374] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:30:09.371541 25553 ts_tablet_manager.cc:1428] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260618 22:30:09.371546 25554 raft_consensus.cc:729] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84e78806f334957b5ecc9f59882f86b, State: Initialized, Role: FOLLOWER
I20260618 22:30:09.372316 25554 consensus_queue.cc:260] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } }
I20260618 22:30:09.373533 25534 heartbeater.cc:503] Master 127.10.242.254:35805 was elected leader, sending a full tablet report...
I20260618 22:30:09.375794 25554 ts_tablet_manager.cc:1428] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260618 22:30:09.414538 25560 raft_consensus.cc:482] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:30:09.414848 25560 raft_consensus.cc:504] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } }
I20260618 22:30:09.416635 25560 leader_election.cc:288] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87cf49fcc4194b5c9429af4a95fc9276 (127.10.242.193:46011), cc33e2b9b6ed4040bc239bfeae5229a4 (127.10.242.194:33811)
I20260618 22:30:09.422091 25536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:09.416888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:46011 (local address 127.10.242.195:38415)
0618 22:30:09.417163 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:09.417183 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:09.417310 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:09.417737 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:09.417742 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:09.417776 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:09.417936 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:09.417944 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.418935 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.418939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:09.420198 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:09.420217 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.421397 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.421403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:09.421568 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:09.421592 (+ 24us) client_negotiation.cc:769] Sending connection context
0618 22:30:09.421674 (+ 82us) client_negotiation.cc:240] Negotiation successful
0618 22:30:09.421812 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260618 22:30:09.422084 25548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:09.417211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:38415 (local address 127.10.242.193:46011)
0618 22:30:09.417414 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:09.417420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:09.417439 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:30:09.417514 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:09.417519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:09.417595 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:09.417685 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:09.418098 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.418795 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.420371 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.421256 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.421398 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:09.421471 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:09.421722 (+ 251us) server_negotiation.cc:299] Negotiation successful
0618 22:30:09.421847 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260618 22:30:09.422808 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:09.417269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:35677 (local address 127.10.242.194:33811)
0618 22:30:09.417472 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:09.417479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:09.418011 (+ 532us) server_negotiation.cc:407] Connection header received
0618 22:30:09.418152 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:09.418156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:09.418215 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:09.418287 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:09.418781 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.419454 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.421068 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.421876 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.422011 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:09.422079 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:09.422479 (+ 400us) server_negotiation.cc:299] Negotiation successful
0618 22:30:09.422590 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260618 22:30:09.423153 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:09.417163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:33811 (local address 127.10.242.195:35677)
0618 22:30:09.417906 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:09.417920 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:09.418031 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:09.418312 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:09.418316 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:09.418338 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:09.418636 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:09.418645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.419594 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.419598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:09.420924 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:09.420934 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.422065 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.422071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:09.422257 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:09.422293 (+ 36us) client_negotiation.cc:769] Sending connection context
0618 22:30:09.422421 (+ 128us) client_negotiation.cc:240] Negotiation successful
0618 22:30:09.422833 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":369,"threads_started":1}
I20260618 22:30:09.423208 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c418a2a4d38a46d79099520da714a5ba" candidate_uuid: "c84e78806f334957b5ecc9f59882f86b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87cf49fcc4194b5c9429af4a95fc9276" is_pre_election: true
I20260618 22:30:09.423803 25222 raft_consensus.cc:2454] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c84e78806f334957b5ecc9f59882f86b in term 0.
I20260618 22:30:09.424247 25356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c418a2a4d38a46d79099520da714a5ba" candidate_uuid: "c84e78806f334957b5ecc9f59882f86b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" is_pre_election: true
I20260618 22:30:09.424535 25422 leader_election.cc:302] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87cf49fcc4194b5c9429af4a95fc9276, c84e78806f334957b5ecc9f59882f86b; no voters:
I20260618 22:30:09.424670 25356 raft_consensus.cc:2454] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c84e78806f334957b5ecc9f59882f86b in term 0.
I20260618 22:30:09.424939 25560 raft_consensus.cc:2781] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:30:09.425107 25560 raft_consensus.cc:482] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:30:09.425225 25560 raft_consensus.cc:3037] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:30:09.427475 25560 raft_consensus.cc:504] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } }
I20260618 22:30:09.427999 25560 leader_election.cc:288] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [CANDIDATE]: Term 1 election: Requested vote from peers 87cf49fcc4194b5c9429af4a95fc9276 (127.10.242.193:46011), cc33e2b9b6ed4040bc239bfeae5229a4 (127.10.242.194:33811)
I20260618 22:30:09.428465 25356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c418a2a4d38a46d79099520da714a5ba" candidate_uuid: "c84e78806f334957b5ecc9f59882f86b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4"
I20260618 22:30:09.428524 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c418a2a4d38a46d79099520da714a5ba" candidate_uuid: "c84e78806f334957b5ecc9f59882f86b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87cf49fcc4194b5c9429af4a95fc9276"
I20260618 22:30:09.428646 25356 raft_consensus.cc:3037] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:30:09.428737 25222 raft_consensus.cc:3037] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:30:09.430883 25356 raft_consensus.cc:2454] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c84e78806f334957b5ecc9f59882f86b in term 1.
I20260618 22:30:09.430912 25222 raft_consensus.cc:2454] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c84e78806f334957b5ecc9f59882f86b in term 1.
I20260618 22:30:09.431293 25423 leader_election.cc:302] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c84e78806f334957b5ecc9f59882f86b, cc33e2b9b6ed4040bc239bfeae5229a4; no voters:
I20260618 22:30:09.431622 25560 raft_consensus.cc:2781] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:30:09.432068 25560 raft_consensus.cc:686] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [term 1 LEADER]: Becoming Leader. State: Replica: c84e78806f334957b5ecc9f59882f86b, State: Running, Role: LEADER
I20260618 22:30:09.432605 25560 consensus_queue.cc:237] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } }
I20260618 22:30:09.437124 25076 catalog_manager.cc:5495] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b reported cstate change: term changed from 0 to 1, leader changed from <none> to c84e78806f334957b5ecc9f59882f86b (127.10.242.195). New cstate: current_term: 1 leader_uuid: "c84e78806f334957b5ecc9f59882f86b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } health_report { overall_health: HEALTHY } } }
I20260618 22:30:09.460145 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:09.454890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57962 (local address 127.10.242.195:42763)
0618 22:30:09.455073 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:09.455077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:09.455095 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:30:09.455243 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:09.455246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:09.455324 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:09.455444 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:30:09.455761 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.456488 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.457959 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.458347 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.458437 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:09.459024 (+ 587us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:30:09.459538 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:30:09.459656 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:09.459798 (+ 142us) server_negotiation.cc:299] Negotiation successful
0618 22:30:09.459871 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:30:09.468494 25356 raft_consensus.cc:1261] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4 [term 1 FOLLOWER]: Refusing update from remote peer c84e78806f334957b5ecc9f59882f86b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:30:09.468492 25222 raft_consensus.cc:1261] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276 [term 1 FOLLOWER]: Refusing update from remote peer c84e78806f334957b5ecc9f59882f86b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:30:09.469494 25562 consensus_queue.cc:1035] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:30:09.469848 25560 consensus_queue.cc:1035] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [LEADER]: Connected to new peer: Peer: permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 22:30:09.477205 25535 tablet.cc:2369] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:30:09.481230 25567 mvcc.cc:205] Tried to move back new op lower bound from 7298342131572985856 to 7298342131440037888. Current Snapshot: MvccSnapshot[applied={T|T < 7298342131572985856}]
I20260618 22:30:09.493848 25076 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:30:09.494248 25076 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:30:09.498219 25076 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e: Sending DeleteTablet for 3 replicas of tablet c418a2a4d38a46d79099520da714a5ba
I20260618 22:30:09.499464 25336 tablet_service.cc:1514] Processing DeleteTablet for tablet c418a2a4d38a46d79099520da714a5ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:30:09 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:35366
I20260618 22:30:09.499480 25202 tablet_service.cc:1514] Processing DeleteTablet for tablet c418a2a4d38a46d79099520da714a5ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:30:09 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:35486
I20260618 22:30:09.499567 25469 tablet_service.cc:1514] Processing DeleteTablet for tablet c418a2a4d38a46d79099520da714a5ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:30:09 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:57954
I20260618 22:30:09.500835 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 25139
I20260618 22:30:09.514887 25575 tablet_replica.cc:330] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4: stopping tablet replica
I20260618 22:30:09.515422 25575 raft_consensus.cc:2229] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:30:09.515986 25575 raft_consensus.cc:2258] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:30:09.518501 25575 ts_tablet_manager.cc:1905] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:30:09.520270 25574 tablet_replica.cc:330] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276: stopping tablet replica
I20260618 22:30:09.520642 25576 tablet_replica.cc:330] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b: stopping tablet replica
I20260618 22:30:09.520735 25574 raft_consensus.cc:2229] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:30:09.521402 25574 raft_consensus.cc:2258] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:30:09.521402 25576 raft_consensus.cc:2229] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:30:09.522217 25576 raft_consensus.cc:2258] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:30:09.524880 25576 ts_tablet_manager.cc:1905] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:30:09.525959 25575 ts_tablet_manager.cc:1918] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:30:09.526129 25575 log.cc:1196] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/c418a2a4d38a46d79099520da714a5ba
I20260618 22:30:09.526677 25575 ts_tablet_manager.cc:1939] T c418a2a4d38a46d79099520da714a5ba P cc33e2b9b6ed4040bc239bfeae5229a4: Deleting consensus metadata
I20260618 22:30:09.527513 25574 ts_tablet_manager.cc:1905] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:30:09.528635 25063 catalog_manager.cc:4841] TS cc33e2b9b6ed4040bc239bfeae5229a4 (127.10.242.194:33811): tablet c418a2a4d38a46d79099520da714a5ba (table test-table [id=3f40acbe3d2146b3bd07a8901c349101]) successfully deleted
Jun 18 22:30:09 dist-test-slave-0956 krb5kdc[25033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:30:09.534157 25576 ts_tablet_manager.cc:1918] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:30:09.534348 25576 log.cc:1196] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/c418a2a4d38a46d79099520da714a5ba
I20260618 22:30:09.534457 25574 ts_tablet_manager.cc:1918] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:30:09.534649 25574 log.cc:1196] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/c418a2a4d38a46d79099520da714a5ba
I20260618 22:30:09.534924 25576 ts_tablet_manager.cc:1939] T c418a2a4d38a46d79099520da714a5ba P c84e78806f334957b5ecc9f59882f86b: Deleting consensus metadata
I20260618 22:30:09.535256 25574 ts_tablet_manager.cc:1939] T c418a2a4d38a46d79099520da714a5ba P 87cf49fcc4194b5c9429af4a95fc9276: Deleting consensus metadata
I20260618 22:30:09.536861 25062 catalog_manager.cc:4841] TS c84e78806f334957b5ecc9f59882f86b (127.10.242.195:42763): tablet c418a2a4d38a46d79099520da714a5ba (table test-table [id=3f40acbe3d2146b3bd07a8901c349101]) successfully deleted
I20260618 22:30:09.537271 25062 catalog_manager.cc:4841] TS 87cf49fcc4194b5c9429af4a95fc9276 (127.10.242.193:46011): tablet c418a2a4d38a46d79099520da714a5ba (table test-table [id=3f40acbe3d2146b3bd07a8901c349101]) successfully deleted
I20260618 22:30:09.540982 25548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:09.502163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35500 (local address 127.10.242.193:46011)
0618 22:30:09.502371 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:09.502379 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:09.517164 (+ 14785us) server_negotiation.cc:407] Connection header received
0618 22:30:09.517444 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:09.517451 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:09.517560 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:09.517640 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:09.521881 (+ 4241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.522992 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.525022 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.525667 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.525771 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:09.533580 (+ 7809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:09.533598 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:09.533616 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:09.533671 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:09.537794 (+ 4123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:09.538579 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:09.538589 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:09.538593 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:09.538752 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:09.539252 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:09.539258 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:09.539262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:09.539753 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:09.540063 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:09.540375 (+ 312us) server_negotiation.cc:299] Negotiation successful
0618 22:30:09.540618 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:30:09.541752 25262 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:09.632942 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 25139
I20260618 22:30:09.653242 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 25273
Jun 18 22:30:09 dist-test-slave-0956 krb5kdc[25033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.194@KRBTEST.COM
I20260618 22:30:09.672022 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:09.654066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35380 (local address 127.10.242.194:33811)
0618 22:30:09.654264 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:09.654268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:09.654286 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:30:09.654502 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:09.654506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:09.654574 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:09.654668 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:09.656206 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.656958 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.659008 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.659631 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.659747 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:09.665058 (+ 5311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:09.665102 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:09.665124 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:09.665172 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:09.668954 (+ 3782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:09.669797 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:09.669807 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:09.669811 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:09.669959 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:09.670348 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:09.670353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:09.670357 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:09.670820 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:09.671174 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:09.671436 (+ 262us) server_negotiation.cc:299] Negotiation successful
0618 22:30:09.671663 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260618 22:30:09.672868 25396 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:09.762876 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 25273
I20260618 22:30:09.782773 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 25406
Jun 18 22:30:09 dist-test-slave-0956 krb5kdc[25033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.195@KRBTEST.COM
I20260618 22:30:09.800452 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:09.783556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57970 (local address 127.10.242.195:42763)
0618 22:30:09.783723 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:09.783727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:09.783782 (+ 55us) server_negotiation.cc:407] Connection header received
0618 22:30:09.784014 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:09.784019 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:09.784089 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:09.784190 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:09.785888 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.786761 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.788671 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:09.789197 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:09.789339 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:09.794759 (+ 5420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:09.794774 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:09.794790 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:09.794836 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:09.797623 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:09.798433 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:09.798440 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:09.798442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:09.798583 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:09.798981 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:09.798984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:09.798986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:09.799347 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:09.799578 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:09.799935 (+ 357us) server_negotiation.cc:299] Negotiation successful
0618 22:30:09.800100 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260618 22:30:09.801117 25529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:09.930291 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 25406
I20260618 22:30:09.945307 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 25046
I20260618 22:30:09.946450 25108 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:10.058866 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 25046
I20260618 22:30:10.076475 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:35805
--webserver_interface=127.10.242.254
--webserver_port=44395
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:35805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:30:10.397483 25582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:10.397840 25582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:10.397989 25582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:10.405344 25582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:30:10.405488 25582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:10.405570 25582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:10.405645 25582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:30:10.405706 25582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:30:10.415829 25582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:35805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:35805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=44395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:10.417270 25582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:10.418853 25582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:10.440510 25588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:10.440488 25589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:10.440456 25591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:10.441380 25582 server_base.cc:1034] running on GCE node
I20260618 22:30:10.442250 25582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:10.443900 25582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:10.445140 25582 hybrid_clock.cc:648] HybridClock initialized: now 1781821810445047 us; error 57 us; skew 500 ppm
I20260618 22:30:10.445578 25582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:10 dist-test-slave-0956 krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821810, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:10.449180 25582 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:30:10.450948 25582 webserver.cc:460] Webserver started at http://127.10.242.254:44395/ using document root <none> and password file <none>
I20260618 22:30:10.451718 25582 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:10.451851 25582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:10.458040 25582 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260618 22:30:10.461361 25598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:10.462797 25582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:30:10.462975 25582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "bffd12ba538c464b868b911063445d7e"
format_stamp: "Formatted at 2026-06-18 22:30:07 on dist-test-slave-0956"
server_key: "d21f3092844ba29bbac61e0c99f86b05"
server_key_iv: "299a75de26f134e15214cd31c48a2637"
server_key_version: "encryptionkey@0"
I20260618 22:30:10.464079 25582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:10.490954 25582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:10.495743 25582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:10.496018 25582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:10.515336 25582 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:35805
I20260618 22:30:10.515362 25649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:35805 every 8 connection(s)
I20260618 22:30:10.516960 25582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:30:10.518493 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 25582
I20260618 22:30:10.519502 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:46011
--local_ip_for_outbound_sockets=127.10.242.193
--tserver_master_addrs=127.10.242.254:35805
--webserver_port=39771
--webserver_interface=127.10.242.193
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:30:10.526374 25650 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:30:10.530045 25650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e: Bootstrap starting.
I20260618 22:30:10.549698 25650 log.cc:824] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e: Log is configured to *not* fsync() on all Append() calls
I20260618 22:30:10.562361 25650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:30:10.562929 25650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e: Bootstrap complete.
I20260618 22:30:10.572496 25650 raft_consensus.cc:348] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffd12ba538c464b868b911063445d7e" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35805 } }
I20260618 22:30:10.573421 25650 raft_consensus.cc:729] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bffd12ba538c464b868b911063445d7e, State: Initialized, Role: FOLLOWER
I20260618 22:30:10.574036 25650 consensus_queue.cc:260] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffd12ba538c464b868b911063445d7e" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35805 } }
I20260618 22:30:10.574260 25650 raft_consensus.cc:388] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:30:10.574378 25650 raft_consensus.cc:482] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:30:10.574545 25650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:30:10.578450 25650 raft_consensus.cc:504] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffd12ba538c464b868b911063445d7e" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35805 } }
I20260618 22:30:10.578989 25650 leader_election.cc:302] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bffd12ba538c464b868b911063445d7e; no voters:
I20260618 22:30:10.579564 25650 leader_election.cc:288] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260618 22:30:10.579710 25654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 2 FOLLOWER]: Leader election won for term 2
I20260618 22:30:10.580822 25654 raft_consensus.cc:686] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 2 LEADER]: Becoming Leader. State: Replica: bffd12ba538c464b868b911063445d7e, State: Running, Role: LEADER
I20260618 22:30:10.581477 25654 consensus_queue.cc:237] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffd12ba538c464b868b911063445d7e" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35805 } }
I20260618 22:30:10.581779 25650 sys_catalog.cc:564] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:30:10.583896 25656 sys_catalog.cc:455] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bffd12ba538c464b868b911063445d7e. Latest consensus state: current_term: 2 leader_uuid: "bffd12ba538c464b868b911063445d7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffd12ba538c464b868b911063445d7e" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35805 } } }
I20260618 22:30:10.584262 25656 sys_catalog.cc:458] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [sys.catalog]: This master's current role is: LEADER
I20260618 22:30:10.584522 25655 sys_catalog.cc:455] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bffd12ba538c464b868b911063445d7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffd12ba538c464b868b911063445d7e" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35805 } } }
I20260618 22:30:10.584841 25655 sys_catalog.cc:458] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [sys.catalog]: This master's current role is: LEADER
I20260618 22:30:10.594648 25662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:30:10.602694 25662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:30:10.604239 25662 catalog_manager.cc:1259] Loaded cluster ID: bf63df4f06ed405ebb7fa01545ceaed8
I20260618 22:30:10.604360 25662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:30:10.607438 25662 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260618 22:30:10.624226 25662 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260618 22:30:10.637944 25662 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:30:10.639730 25662 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e: Loaded TSK: 0
I20260618 22:30:10.641413 25662 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260618 22:30:10.884406 25652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:10.884711 25652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:10.884847 25652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:10.892313 25652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:10.892458 25652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:10.892617 25652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:30:10.902876 25652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:46011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=39771
--tserver_master_addrs=127.10.242.254:35805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:10.904204 25652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:10.905604 25652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:10.920404 25680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:10.920608 25677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:10.920854 25678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:10.921360 25652 server_base.cc:1034] running on GCE node
I20260618 22:30:10.922176 25652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:10.923542 25652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:10.924810 25652 hybrid_clock.cc:648] HybridClock initialized: now 1781821810924743 us; error 53 us; skew 500 ppm
I20260618 22:30:10.925259 25652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:10 dist-test-slave-0956 krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821810, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:10.929615 25652 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:30:10.931605 25652 webserver.cc:460] Webserver started at http://127.10.242.193:39771/ using document root <none> and password file <none>
I20260618 22:30:10.932591 25652 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:10.932766 25652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:10.940740 25652 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:30:10.944715 25687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:10.946678 25652 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260618 22:30:10.946907 25652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "87cf49fcc4194b5c9429af4a95fc9276"
format_stamp: "Formatted at 2026-06-18 22:30:08 on dist-test-slave-0956"
server_key: "ba3695311fb55c4c288cc1c013c54009"
server_key_iv: "a5576b23f636a677ca4993fc06de6ca5"
server_key_version: "encryptionkey@0"
I20260618 22:30:10.948360 25652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:10.980732 25652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:10.986389 25652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:10.986689 25652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:10.988005 25652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:30:10.989837 25652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:30:10.989951 25652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:10.990057 25652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:30:10.990124 25652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:11.022574 25652 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:46011
I20260618 22:30:11.022629 25799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:46011 every 8 connection(s)
I20260618 22:30:11.024562 25652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:30:11.026397 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 25652
I20260618 22:30:11.027052 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:33811
--local_ip_for_outbound_sockets=127.10.242.194
--tserver_master_addrs=127.10.242.254:35805
--webserver_port=43495
--webserver_interface=127.10.242.194
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:30:11 dist-test-slave-0956 krb5kdc[25033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821810, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:30:11.050282 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:11.027819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:48249 (local address 127.10.242.254:35805)
0618 22:30:11.028542 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:11.028562 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:11.029382 (+ 820us) server_negotiation.cc:407] Connection header received
0618 22:30:11.031656 (+ 2274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:11.031676 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:11.032183 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:11.032633 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:11.034510 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.035796 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.037479 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.038147 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.038327 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:11.041822 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:11.041830 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:11.041854 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:11.041894 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:11.045397 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:11.046301 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:11.046309 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:11.046315 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:11.046503 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:11.046973 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:11.046977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:11.046979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:11.047475 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:11.047668 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:11.048582 (+ 914us) server_negotiation.cc:299] Negotiation successful
0618 22:30:11.048990 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":282,"threads_started":1}
I20260618 22:30:11.050283 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:11.027984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:48249)
0618 22:30:11.029176 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:11.029228 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:11.030993 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:11.032779 (+ 1786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:11.032787 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:11.033445 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:11.034235 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:11.034262 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.036008 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.036012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:11.037329 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:11.037351 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.038396 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.038403 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:11.038660 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:11.039316 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:11.039380 (+ 64us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:11.041532 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:11.045584 (+ 4052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:11.045592 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:11.045606 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:11.046121 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:11.046628 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:11.046632 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:11.046635 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:11.046849 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:11.047621 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:11.047630 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:11.048045 (+ 415us) client_negotiation.cc:769] Sending connection context
0618 22:30:11.048320 (+ 275us) client_negotiation.cc:240] Negotiation successful
0618 22:30:11.048901 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":879,"thread_start_us":674,"threads_started":1}
I20260618 22:30:11.053570 25800 heartbeater.cc:347] Connected to a master server at 127.10.242.254:35805
I20260618 22:30:11.053932 25800 heartbeater.cc:464] Registering TS with master...
I20260618 22:30:11.054843 25800 heartbeater.cc:511] Master 127.10.242.254:35805 requested a full tablet report, sending...
I20260618 22:30:11.057165 25615 ts_manager.cc:194] Registered new tserver with Master: 87cf49fcc4194b5c9429af4a95fc9276 (127.10.242.193:46011)
I20260618 22:30:11.058586 25615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:48249
W20260618 22:30:11.349056 25804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:11.349349 25804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:11.349457 25804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:11.356567 25804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:11.356685 25804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:11.356853 25804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:30:11.367100 25804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:33811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=43495
--tserver_master_addrs=127.10.242.254:35805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:11.368508 25804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:11.369944 25804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:11.382871 25810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:11.382834 25809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:11.382931 25812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:11.383550 25804 server_base.cc:1034] running on GCE node
I20260618 22:30:11.384130 25804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:11.385061 25804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:11.386298 25804 hybrid_clock.cc:648] HybridClock initialized: now 1781821811386238 us; error 53 us; skew 500 ppm
I20260618 22:30:11.386613 25804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:11 dist-test-slave-0956 krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821811, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:11.389710 25804 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:30:11.391132 25804 webserver.cc:460] Webserver started at http://127.10.242.194:43495/ using document root <none> and password file <none>
I20260618 22:30:11.391793 25804 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:11.391923 25804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:11.397311 25804 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:30:11.400005 25819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:11.401472 25804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:30:11.401635 25804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "cc33e2b9b6ed4040bc239bfeae5229a4"
format_stamp: "Formatted at 2026-06-18 22:30:08 on dist-test-slave-0956"
server_key: "39c155f6c1d3099b8e88140b6b5c04a5"
server_key_iv: "c6c2a8e75ba9ba4ba9504114c02fc366"
server_key_version: "encryptionkey@0"
I20260618 22:30:11.402580 25804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:11.421820 25804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:11.426082 25804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:11.426348 25804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:11.427289 25804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:30:11.428617 25804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:30:11.428727 25804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:11.428839 25804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:30:11.428902 25804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:11.459260 25804 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:33811
I20260618 22:30:11.459349 25931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:33811 every 8 connection(s)
I20260618 22:30:11.460671 25804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:30:11.469729 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 25804
I20260618 22:30:11.470296 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:42763
--local_ip_for_outbound_sockets=127.10.242.195
--tserver_master_addrs=127.10.242.254:35805
--webserver_port=38425
--webserver_interface=127.10.242.195
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:30:11 dist-test-slave-0956 krb5kdc[25033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821811, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:30:11.479337 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:11.463085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:44575 (local address 127.10.242.254:35805)
0618 22:30:11.463261 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:11.463265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:11.464338 (+ 1073us) server_negotiation.cc:407] Connection header received
0618 22:30:11.466239 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:11.466244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:11.466323 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:11.466407 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:11.467931 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.468640 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.470253 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.470722 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.470834 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:11.474082 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:11.474088 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:11.474091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:11.474124 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:11.476461 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:11.477153 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:11.477159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:11.477163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:11.477283 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:11.477679 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:11.477684 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:11.477686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:11.477962 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:11.478092 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:11.478769 (+ 677us) server_negotiation.cc:299] Negotiation successful
0618 22:30:11.479020 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260618 22:30:11.480290 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:11.463492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:44575)
0618 22:30:11.464180 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:11.464219 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:11.465971 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:11.466688 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:11.466697 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:11.467178 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:11.467752 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:11.467769 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.468789 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.468793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:11.470112 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:11.470124 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.470927 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.470934 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:11.471169 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:11.471769 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:11.471791 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:11.473856 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:11.476602 (+ 2746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:11.476610 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:11.476628 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:11.477019 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:11.477389 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:11.477394 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:11.477398 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:11.477571 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:11.478086 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:11.478094 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:11.478422 (+ 328us) client_negotiation.cc:769] Sending connection context
0618 22:30:11.478651 (+ 229us) client_negotiation.cc:240] Negotiation successful
0618 22:30:11.479152 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":263,"threads_started":1}
I20260618 22:30:11.481724 25932 heartbeater.cc:347] Connected to a master server at 127.10.242.254:35805
I20260618 22:30:11.482059 25932 heartbeater.cc:464] Registering TS with master...
I20260618 22:30:11.482879 25932 heartbeater.cc:511] Master 127.10.242.254:35805 requested a full tablet report, sending...
I20260618 22:30:11.484439 25615 ts_manager.cc:194] Registered new tserver with Master: cc33e2b9b6ed4040bc239bfeae5229a4 (127.10.242.194:33811)
I20260618 22:30:11.485585 25615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:44575
W20260618 22:30:11.785923 25935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:11.786190 25935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:11.786348 25935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:11.793447 25935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:11.793553 25935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:11.793669 25935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:30:11.803176 25935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:42763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=38425
--tserver_master_addrs=127.10.242.254:35805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:11.804373 25935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:11.805606 25935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:11.818214 25940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:11.818255 25941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:11.818275 25943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:11.819175 25935 server_base.cc:1034] running on GCE node
I20260618 22:30:11.819789 25935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:11.820783 25935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:11.821981 25935 hybrid_clock.cc:648] HybridClock initialized: now 1781821811821912 us; error 63 us; skew 500 ppm
I20260618 22:30:11.822292 25935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:11 dist-test-slave-0956 krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821811, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:11.825495 25935 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:30:11.826999 25935 webserver.cc:460] Webserver started at http://127.10.242.195:38425/ using document root <none> and password file <none>
I20260618 22:30:11.827678 25935 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:11.827818 25935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:11.833194 25935 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260618 22:30:11.835959 25950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:11.837487 25935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260618 22:30:11.837656 25935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "c84e78806f334957b5ecc9f59882f86b"
format_stamp: "Formatted at 2026-06-18 22:30:09 on dist-test-slave-0956"
server_key: "9c1afa46cc3be1eb2de26465ec16843d"
server_key_iv: "6567709e029397ceb1ed94afa45fbb39"
server_key_version: "encryptionkey@0"
I20260618 22:30:11.838641 25935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:11.859468 25935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:11.863709 25935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:11.863938 25935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:11.864893 25935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:30:11.866238 25935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:30:11.866325 25935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:11.866415 25935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:30:11.866474 25935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:11.896440 25935 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:42763
I20260618 22:30:11.896510 26062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:42763 every 8 connection(s)
I20260618 22:30:11.897903 25935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:30:11.902045 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 25935
Jun 18 22:30:11 dist-test-slave-0956 krb5kdc[25033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821811, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:30:11.917663 26067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:11.903022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38128 (local address 127.10.242.254:35805)
0618 22:30:11.903922 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:11.903926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:11.903939 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:30:11.903988 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:11.903991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:11.904045 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:11.904135 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:11.905752 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.906592 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.908022 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.908432 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.908537 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:11.911313 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:11.911318 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:11.911321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:11.911351 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:11.915124 (+ 3773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:11.915756 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:11.915761 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:11.915765 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:11.915936 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:11.916301 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:11.916306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:11.916309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:11.916659 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:11.916926 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:11.917176 (+ 250us) server_negotiation.cc:299] Negotiation successful
0618 22:30:11.917342 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":800,"spinlock_wait_cycles":4864,"thread_start_us":690,"threads_started":1}
I20260618 22:30:11.918809 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:11.900285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:53939 (local address 127.10.242.254:35805)
0618 22:30:11.900483 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:11.900487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:11.901789 (+ 1302us) server_negotiation.cc:407] Connection header received
0618 22:30:11.904342 (+ 2553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:11.904355 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:11.904435 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:11.904528 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:11.906239 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.907082 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.908675 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.909305 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.909434 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:11.912977 (+ 3543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:11.912982 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:11.912985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:11.913028 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:11.915734 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:11.916472 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:11.916475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:11.916477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:11.916595 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:11.917036 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:11.917041 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:11.917044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:11.917403 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:11.917537 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:11.918272 (+ 735us) server_negotiation.cc:299] Negotiation successful
0618 22:30:11.918508 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260618 22:30:11.919783 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:11.900762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:53939)
0618 22:30:11.901595 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:11.901648 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:11.904055 (+ 2407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:11.904793 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:11.904803 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:11.905480 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:11.906068 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:11.906088 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.907261 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.907265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:11.908526 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:11.908539 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.909572 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.909580 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:11.909813 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:11.910462 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:11.910493 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:11.912761 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:11.915864 (+ 3103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:11.915873 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:11.915889 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:11.916349 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:11.916689 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:11.916693 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:11.916697 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:11.916925 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:11.917531 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:11.917539 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:11.917946 (+ 407us) client_negotiation.cc:769] Sending connection context
0618 22:30:11.918173 (+ 227us) client_negotiation.cc:240] Negotiation successful
0618 22:30:11.918608 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":311,"threads_started":1}
I20260618 22:30:11.920986 26063 heartbeater.cc:347] Connected to a master server at 127.10.242.254:35805
I20260618 22:30:11.921278 26063 heartbeater.cc:464] Registering TS with master...
I20260618 22:30:11.921949 26063 heartbeater.cc:511] Master 127.10.242.254:35805 requested a full tablet report, sending...
I20260618 22:30:11.923282 25615 ts_manager.cc:194] Registered new tserver with Master: c84e78806f334957b5ecc9f59882f86b (127.10.242.195:42763)
I20260618 22:30:11.924069 25615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:53939
I20260618 22:30:11.932462 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:30:11.939070 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:11.933506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38138 (local address 127.10.242.254:35805)
0618 22:30:11.933656 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:11.933661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:11.934067 (+ 406us) server_negotiation.cc:407] Connection header received
0618 22:30:11.934282 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:11.934286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:11.934422 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:11.934537 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:30:11.934964 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.935684 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.937237 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.937630 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.937720 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:11.938168 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:30:11.938647 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:30:11.938747 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:11.938785 (+ 38us) server_negotiation.cc:299] Negotiation successful
0618 22:30:11.938846 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:30:11.939950 25615 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:38138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:30:11.947212 25615 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:30:11.979831 26073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:11.959020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:33811 (local address 127.0.0.1:35396)
0618 22:30:11.970741 (+ 11721us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:11.970760 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:11.970875 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:11.972204 (+ 1329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:11.972212 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:11.972240 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:11.972756 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:11.972770 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.974507 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.974513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:11.977021 (+ 2508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:11.977032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.978730 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.978738 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:11.978929 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:11.978990 (+ 61us) client_negotiation.cc:769] Sending connection context
0618 22:30:11.979130 (+ 140us) client_negotiation.cc:240] Negotiation successful
0618 22:30:11.979353 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11586,"mutex_wait_us":65,"thread_start_us":10290,"threads_started":1}
I20260618 22:30:11.980007 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:11.958874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35514 (local address 127.10.242.193:46011)
0618 22:30:11.972407 (+ 13533us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:11.972417 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:11.972446 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:30:11.972541 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:11.972548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:11.972769 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:11.973010 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:11.973488 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.974917 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.977602 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.978975 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.979124 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:11.979287 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:11.979546 (+ 259us) server_negotiation.cc:299] Negotiation successful
0618 22:30:11.979716 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13361,"thread_start_us":13205,"threads_started":1}
I20260618 22:30:11.980038 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:11.958660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:46011 (local address 127.0.0.1:35514)
0618 22:30:11.969710 (+ 11050us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:11.969760 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:11.969942 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:11.972991 (+ 3049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:11.972995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:11.973020 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:11.973333 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:11.973342 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.975130 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.975134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:11.977431 (+ 2297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:11.977447 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.979186 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.979192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:11.979362 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:11.979394 (+ 32us) client_negotiation.cc:769] Sending connection context
0618 22:30:11.979465 (+ 71us) client_negotiation.cc:240] Negotiation successful
0618 22:30:11.979591 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10774,"mutex_wait_us":41,"thread_start_us":11777,"threads_started":1}
I20260618 22:30:11.980837 26076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:11.959496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35396 (local address 127.10.242.194:33811)
0618 22:30:11.971510 (+ 12014us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:11.971520 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:11.971559 (+ 39us) server_negotiation.cc:407] Connection header received
0618 22:30:11.971671 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:11.971681 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:11.971960 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:11.972978 (+ 1018us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:11.973229 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.974358 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.977167 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.978503 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.978687 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:11.978861 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:11.980366 (+ 1505us) server_negotiation.cc:299] Negotiation successful
0618 22:30:11.980526 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11709,"thread_start_us":11767,"threads_started":1}
I20260618 22:30:11.982185 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:11.971114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57986 (local address 127.10.242.195:42763)
0618 22:30:11.971938 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:11.971946 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:11.972881 (+ 935us) server_negotiation.cc:407] Connection header received
0618 22:30:11.973551 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:11.973559 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:11.973772 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:11.973958 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:11.975114 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.976727 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.978947 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.980697 (+ 1750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.980895 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:11.981273 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:11.981741 (+ 468us) server_negotiation.cc:299] Negotiation successful
0618 22:30:11.981900 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":531,"threads_started":1}
I20260618 22:30:11.982214 26078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:11.971932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:42763 (local address 127.0.0.1:57986)
0618 22:30:11.972752 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:11.972769 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:11.972905 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:11.973969 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:11.973974 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:11.974010 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:11.974341 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:11.974350 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.976868 (+ 2518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.976874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:11.978748 (+ 1874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:11.978762 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:11.980929 (+ 2167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:11.980937 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:11.981541 (+ 604us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:11.981579 (+ 38us) client_negotiation.cc:769] Sending connection context
0618 22:30:11.981684 (+ 105us) client_negotiation.cc:240] Negotiation successful
0618 22:30:11.981814 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":40,"thread_start_us":531,"threads_started":1}
I20260618 22:30:11.984032 25735 tablet_service.cc:1467] Processing CreateTablet for tablet 585260811dc54d5797da1f1fde81ea2d (DEFAULT_TABLE table=test-table [id=f69f7ebcd4ad4b5ba8a653a18dbc70c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:30:11.984529 25867 tablet_service.cc:1467] Processing CreateTablet for tablet 585260811dc54d5797da1f1fde81ea2d (DEFAULT_TABLE table=test-table [id=f69f7ebcd4ad4b5ba8a653a18dbc70c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:30:11.985265 25735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 585260811dc54d5797da1f1fde81ea2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:30:11.985823 25867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 585260811dc54d5797da1f1fde81ea2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:30:11.985621 25998 tablet_service.cc:1467] Processing CreateTablet for tablet 585260811dc54d5797da1f1fde81ea2d (DEFAULT_TABLE table=test-table [id=f69f7ebcd4ad4b5ba8a653a18dbc70c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:30:11.986881 25998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 585260811dc54d5797da1f1fde81ea2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:30:11.997977 26079 tablet_bootstrap.cc:492] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276: Bootstrap starting.
I20260618 22:30:12.000538 26080 tablet_bootstrap.cc:492] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4: Bootstrap starting.
I20260618 22:30:12.002050 26081 tablet_bootstrap.cc:492] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b: Bootstrap starting.
I20260618 22:30:12.003648 26079 tablet_bootstrap.cc:654] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276: Neither blocks nor log segments found. Creating new log.
I20260618 22:30:12.004377 26080 tablet_bootstrap.cc:654] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4: Neither blocks nor log segments found. Creating new log.
I20260618 22:30:12.004760 26081 tablet_bootstrap.cc:654] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b: Neither blocks nor log segments found. Creating new log.
I20260618 22:30:12.005039 26079 log.cc:824] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276: Log is configured to *not* fsync() on all Append() calls
I20260618 22:30:12.005712 26080 log.cc:824] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4: Log is configured to *not* fsync() on all Append() calls
I20260618 22:30:12.005774 26081 log.cc:824] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:30:12.008507 26079 tablet_bootstrap.cc:492] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276: No bootstrap required, opened a new log
I20260618 22:30:12.008790 26079 ts_tablet_manager.cc:1397] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260618 22:30:12.008841 26081 tablet_bootstrap.cc:492] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b: No bootstrap required, opened a new log
I20260618 22:30:12.009125 26081 ts_tablet_manager.cc:1397] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260618 22:30:12.009601 26080 tablet_bootstrap.cc:492] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4: No bootstrap required, opened a new log
I20260618 22:30:12.009920 26080 ts_tablet_manager.cc:1397] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260618 22:30:12.016911 26079 raft_consensus.cc:348] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } }
I20260618 22:30:12.017345 26079 raft_consensus.cc:374] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:30:12.017452 26079 raft_consensus.cc:729] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87cf49fcc4194b5c9429af4a95fc9276, State: Initialized, Role: FOLLOWER
I20260618 22:30:12.018065 26079 consensus_queue.cc:260] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } }
I20260618 22:30:12.019323 25800 heartbeater.cc:503] Master 127.10.242.254:35805 was elected leader, sending a full tablet report...
I20260618 22:30:12.020179 26079 ts_tablet_manager.cc:1428] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260618 22:30:12.019933 26080 raft_consensus.cc:348] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } }
I20260618 22:30:12.019999 26081 raft_consensus.cc:348] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } }
I20260618 22:30:12.020358 26080 raft_consensus.cc:374] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:30:12.020358 26081 raft_consensus.cc:374] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:30:12.020454 26081 raft_consensus.cc:729] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84e78806f334957b5ecc9f59882f86b, State: Initialized, Role: FOLLOWER
I20260618 22:30:12.020452 26080 raft_consensus.cc:729] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc33e2b9b6ed4040bc239bfeae5229a4, State: Initialized, Role: FOLLOWER
I20260618 22:30:12.021050 26081 consensus_queue.cc:260] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } }
I20260618 22:30:12.021052 26080 consensus_queue.cc:260] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } }
W20260618 22:30:12.027014 25801 tablet.cc:2369] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:30:12.029120 25932 heartbeater.cc:503] Master 127.10.242.254:35805 was elected leader, sending a full tablet report...
I20260618 22:30:12.029325 26081 ts_tablet_manager.cc:1428] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b: Time spent starting tablet: real 0.020s user 0.015s sys 0.003s
I20260618 22:30:12.029685 26080 ts_tablet_manager.cc:1428] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4: Time spent starting tablet: real 0.020s user 0.013s sys 0.006s
I20260618 22:30:12.030624 26063 heartbeater.cc:503] Master 127.10.242.254:35805 was elected leader, sending a full tablet report...
W20260618 22:30:12.149427 26064 tablet.cc:2369] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:30:12.212532 25933 tablet.cc:2369] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:30:12.304031 26087 raft_consensus.cc:482] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:30:12.304311 26087 raft_consensus.cc:504] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } }
I20260618 22:30:12.305999 26087 leader_election.cc:288] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87cf49fcc4194b5c9429af4a95fc9276 (127.10.242.193:46011), c84e78806f334957b5ecc9f59882f86b (127.10.242.195:42763)
I20260618 22:30:12.314054 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:12.306330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:49769 (local address 127.10.242.193:46011)
0618 22:30:12.306637 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:12.306644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:12.307047 (+ 403us) server_negotiation.cc:407] Connection header received
0618 22:30:12.307252 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:12.307269 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:12.307367 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:12.307436 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:12.308133 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:12.309186 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:12.311706 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:12.312923 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:12.313103 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:12.313179 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:12.313669 (+ 490us) server_negotiation.cc:299] Negotiation successful
0618 22:30:12.313792 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260618 22:30:12.314054 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:12.307197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:44057 (local address 127.10.242.195:42763)
0618 22:30:12.307408 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:12.307412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:12.307559 (+ 147us) server_negotiation.cc:407] Connection header received
0618 22:30:12.307707 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:12.307711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:12.307771 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:12.307847 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:12.308470 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:12.309455 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:12.311750 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:12.312982 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:12.313144 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:12.313220 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:12.313661 (+ 441us) server_negotiation.cc:299] Negotiation successful
0618 22:30:12.313790 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260618 22:30:12.314234 26089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:12.306932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:42763 (local address 127.10.242.194:44057)
0618 22:30:12.307477 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:12.307493 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:12.307605 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:12.307896 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:12.307904 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:12.307928 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:12.308261 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:12.308270 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:12.309565 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:12.309570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:12.311587 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:12.311597 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:12.313211 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:12.313216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:12.313461 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:12.313529 (+ 68us) client_negotiation.cc:769] Sending connection context
0618 22:30:12.313622 (+ 93us) client_negotiation.cc:240] Negotiation successful
0618 22:30:12.313784 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":78,"thread_start_us":257,"threads_started":1}
I20260618 22:30:12.314425 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:12.306216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:46011 (local address 127.10.242.194:49769)
0618 22:30:12.306825 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:12.306845 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:12.307056 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:12.307634 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:12.307639 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:12.307660 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:12.307975 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:12.307987 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:12.309309 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:12.309312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:12.311540 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:12.311555 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:12.313189 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:12.313199 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:12.313461 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:12.313537 (+ 76us) client_negotiation.cc:769] Sending connection context
0618 22:30:12.313619 (+ 82us) client_negotiation.cc:240] Negotiation successful
0618 22:30:12.313784 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"mutex_wait_us":56,"thread_start_us":327,"threads_started":1}
I20260618 22:30:12.315444 26018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585260811dc54d5797da1f1fde81ea2d" candidate_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84e78806f334957b5ecc9f59882f86b" is_pre_election: true
I20260618 22:30:12.315407 25755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585260811dc54d5797da1f1fde81ea2d" candidate_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87cf49fcc4194b5c9429af4a95fc9276" is_pre_election: true
I20260618 22:30:12.315997 26018 raft_consensus.cc:2454] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc33e2b9b6ed4040bc239bfeae5229a4 in term 0.
I20260618 22:30:12.316084 25755 raft_consensus.cc:2454] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc33e2b9b6ed4040bc239bfeae5229a4 in term 0.
I20260618 22:30:12.316623 25820 leader_election.cc:302] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c84e78806f334957b5ecc9f59882f86b, cc33e2b9b6ed4040bc239bfeae5229a4; no voters:
I20260618 22:30:12.316992 26087 raft_consensus.cc:2781] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:30:12.317159 26087 raft_consensus.cc:482] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:30:12.317273 26087 raft_consensus.cc:3037] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:30:12.319429 26087 raft_consensus.cc:504] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } }
I20260618 22:30:12.320077 26087 leader_election.cc:288] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [CANDIDATE]: Term 1 election: Requested vote from peers 87cf49fcc4194b5c9429af4a95fc9276 (127.10.242.193:46011), c84e78806f334957b5ecc9f59882f86b (127.10.242.195:42763)
I20260618 22:30:12.320564 25755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585260811dc54d5797da1f1fde81ea2d" candidate_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87cf49fcc4194b5c9429af4a95fc9276"
I20260618 22:30:12.320644 26018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585260811dc54d5797da1f1fde81ea2d" candidate_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84e78806f334957b5ecc9f59882f86b"
I20260618 22:30:12.320778 25755 raft_consensus.cc:3037] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:30:12.320870 26018 raft_consensus.cc:3037] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:30:12.322965 25755 raft_consensus.cc:2454] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc33e2b9b6ed4040bc239bfeae5229a4 in term 1.
I20260618 22:30:12.323047 26018 raft_consensus.cc:2454] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc33e2b9b6ed4040bc239bfeae5229a4 in term 1.
I20260618 22:30:12.323406 25820 leader_election.cc:302] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87cf49fcc4194b5c9429af4a95fc9276, cc33e2b9b6ed4040bc239bfeae5229a4; no voters:
I20260618 22:30:12.323751 26087 raft_consensus.cc:2781] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:30:12.324162 26087 raft_consensus.cc:686] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [term 1 LEADER]: Becoming Leader. State: Replica: cc33e2b9b6ed4040bc239bfeae5229a4, State: Running, Role: LEADER
I20260618 22:30:12.324630 26087 consensus_queue.cc:237] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } }
I20260618 22:30:12.329056 25615 catalog_manager.cc:5495] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc33e2b9b6ed4040bc239bfeae5229a4 (127.10.242.194). New cstate: current_term: 1 leader_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc33e2b9b6ed4040bc239bfeae5229a4" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33811 } health_report { overall_health: HEALTHY } } }
I20260618 22:30:12.429476 26076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:12.424354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35398 (local address 127.10.242.194:33811)
0618 22:30:12.424587 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:12.424593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:12.424611 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:30:12.424781 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:12.424785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:12.424865 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:12.425002 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:30:12.425403 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:12.426091 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:12.427432 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:12.427831 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:12.427940 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:12.428399 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:30:12.428894 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:30:12.429034 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:12.429140 (+ 106us) server_negotiation.cc:299] Negotiation successful
0618 22:30:12.429221 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260618 22:30:12.437239 26018 raft_consensus.cc:1261] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b [term 1 FOLLOWER]: Refusing update from remote peer cc33e2b9b6ed4040bc239bfeae5229a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:30:12.437285 25755 raft_consensus.cc:1261] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276 [term 1 FOLLOWER]: Refusing update from remote peer cc33e2b9b6ed4040bc239bfeae5229a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:30:12.438139 26090 consensus_queue.cc:1035] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87cf49fcc4194b5c9429af4a95fc9276" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 46011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:30:12.438490 26087 consensus_queue.cc:1035] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c84e78806f334957b5ecc9f59882f86b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:30:12.473022 25615 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:38138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:30:12.473330 25615 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:38138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:30:12.477239 25615 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e: Sending DeleteTablet for 3 replicas of tablet 585260811dc54d5797da1f1fde81ea2d
I20260618 22:30:12.478529 25735 tablet_service.cc:1514] Processing DeleteTablet for tablet 585260811dc54d5797da1f1fde81ea2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:30:12 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:35514
I20260618 22:30:12.478713 25998 tablet_service.cc:1514] Processing DeleteTablet for tablet 585260811dc54d5797da1f1fde81ea2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:30:12 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:57986
I20260618 22:30:12.479231 25867 tablet_service.cc:1514] Processing DeleteTablet for tablet 585260811dc54d5797da1f1fde81ea2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:30:12 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:35396
I20260618 22:30:12.479915 26103 tablet_replica.cc:330] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276: stopping tablet replica
I20260618 22:30:12.480263 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 25652
I20260618 22:30:12.480607 26103 raft_consensus.cc:2229] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:30:12.481335 26103 raft_consensus.cc:2258] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:30:12.482124 26104 tablet_replica.cc:330] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b: stopping tablet replica
I20260618 22:30:12.482543 26104 raft_consensus.cc:2229] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:30:12.483055 26104 raft_consensus.cc:2258] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:30:12.485208 26103 ts_tablet_manager.cc:1905] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:30:12.487931 26104 ts_tablet_manager.cc:1905] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:30:12.492022 26103 ts_tablet_manager.cc:1918] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:30:12.492187 26103 log.cc:1196] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/585260811dc54d5797da1f1fde81ea2d
I20260618 22:30:12.492712 26103 ts_tablet_manager.cc:1939] T 585260811dc54d5797da1f1fde81ea2d P 87cf49fcc4194b5c9429af4a95fc9276: Deleting consensus metadata
I20260618 22:30:12.494699 25599 catalog_manager.cc:4841] TS 87cf49fcc4194b5c9429af4a95fc9276 (127.10.242.193:46011): tablet 585260811dc54d5797da1f1fde81ea2d (table test-table [id=f69f7ebcd4ad4b5ba8a653a18dbc70c3]) successfully deleted
I20260618 22:30:12.497105 26104 ts_tablet_manager.cc:1918] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:30:12.497364 26104 log.cc:1196] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/585260811dc54d5797da1f1fde81ea2d
I20260618 22:30:12.497431 26105 tablet_replica.cc:330] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4: stopping tablet replica
I20260618 22:30:12.497928 26105 raft_consensus.cc:2229] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:30:12.497982 26104 ts_tablet_manager.cc:1939] T 585260811dc54d5797da1f1fde81ea2d P c84e78806f334957b5ecc9f59882f86b: Deleting consensus metadata
I20260618 22:30:12.498620 26105 raft_consensus.cc:2258] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:30:12.499975 25599 catalog_manager.cc:4841] TS c84e78806f334957b5ecc9f59882f86b (127.10.242.195:42763): tablet 585260811dc54d5797da1f1fde81ea2d (table test-table [id=f69f7ebcd4ad4b5ba8a653a18dbc70c3]) successfully deleted
I20260618 22:30:12.500646 26105 ts_tablet_manager.cc:1905] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:30:12.502205 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:12.481331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35520 (local address 127.10.242.193:46011)
0618 22:30:12.481535 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:12.481541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:12.483934 (+ 2393us) server_negotiation.cc:407] Connection header received
0618 22:30:12.484092 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:12.484097 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:12.484174 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:12.484263 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:12.487010 (+ 2747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:12.488081 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:12.490011 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:12.490683 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:12.490781 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:12.494843 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:12.494857 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:12.494876 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:12.494919 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:12.498835 (+ 3916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:12.499893 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:12.499903 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:12.499906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:12.500066 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:12.500526 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:12.500531 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:12.500534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:12.500973 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:12.501299 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:12.501627 (+ 328us) server_negotiation.cc:299] Negotiation successful
0618 22:30:12.501841 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260618 22:30:12.502835 25795 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:12.509218 26105 ts_tablet_manager.cc:1918] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:30:12.509419 26105 log.cc:1196] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/585260811dc54d5797da1f1fde81ea2d
I20260618 22:30:12.510025 26105 ts_tablet_manager.cc:1939] T 585260811dc54d5797da1f1fde81ea2d P cc33e2b9b6ed4040bc239bfeae5229a4: Deleting consensus metadata
I20260618 22:30:12.511972 25600 catalog_manager.cc:4841] TS cc33e2b9b6ed4040bc239bfeae5229a4 (127.10.242.194:33811): tablet 585260811dc54d5797da1f1fde81ea2d (table test-table [id=f69f7ebcd4ad4b5ba8a653a18dbc70c3]) successfully deleted
I20260618 22:30:12.606598 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 25652
I20260618 22:30:12.627847 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 25804
I20260618 22:30:12.645295 26076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:12.628688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35414 (local address 127.10.242.194:33811)
0618 22:30:12.628891 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:12.628896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:12.628916 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:30:12.629036 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:12.629042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:12.629140 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:12.629225 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:12.630512 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:12.631310 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:12.632587 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:12.633044 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:12.633204 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:12.638804 (+ 5600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:12.638825 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:12.638848 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:12.638903 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:12.642515 (+ 3612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:12.643208 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:12.643216 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:12.643218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:12.643341 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:12.643731 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:12.643735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:12.643737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:12.644098 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:12.644419 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:12.644708 (+ 289us) server_negotiation.cc:299] Negotiation successful
0618 22:30:12.644924 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:30:12.646010 25927 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:12.758374 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 25804
I20260618 22:30:12.773607 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 25935
I20260618 22:30:12.787147 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:12.774401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58000 (local address 127.10.242.195:42763)
0618 22:30:12.774598 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:12.774603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:12.774639 (+ 36us) server_negotiation.cc:407] Connection header received
0618 22:30:12.774844 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:12.774848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:12.774914 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:12.774996 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:12.776318 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:12.777025 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:12.778314 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:12.778711 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:12.778830 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:12.781469 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:12.781483 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:12.781499 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:12.781553 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:12.784430 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:12.785136 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:12.785146 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:12.785150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:12.785303 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:12.785662 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:12.785668 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:12.785672 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:12.786035 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:12.786306 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:12.786608 (+ 302us) server_negotiation.cc:299] Negotiation successful
0618 22:30:12.786813 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260618 22:30:12.787739 26058 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:12.877138 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 25935
I20260618 22:30:12.890302 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 25582
I20260618 22:30:12.891515 25645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:13.002674 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 25582
I20260618 22:30:13.020381 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:35805
--webserver_interface=127.10.242.254
--webserver_port=44395
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:35805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:30:13.337961 26111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:13.338248 26111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:13.338335 26111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:13.345948 26111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:30:13.346077 26111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:13.346127 26111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:13.346182 26111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:30:13.346225 26111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:30:13.356187 26111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:35805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:35805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=44395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:13.357471 26111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:13.358708 26111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:13.370175 26116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:13.370214 26117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:13.370788 26119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:13.371030 26111 server_base.cc:1034] running on GCE node
I20260618 22:30:13.371841 26111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:13.373379 26111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:13.374595 26111 hybrid_clock.cc:648] HybridClock initialized: now 1781821813374537 us; error 44 us; skew 500 ppm
I20260618 22:30:13.374984 26111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:13 dist-test-slave-0956 krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821813, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:13.378180 26111 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:30:13.379704 26111 webserver.cc:460] Webserver started at http://127.10.242.254:44395/ using document root <none> and password file <none>
I20260618 22:30:13.380395 26111 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:13.380507 26111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:13.386035 26111 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260618 22:30:13.388782 26126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:13.390321 26111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:30:13.390501 26111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "bffd12ba538c464b868b911063445d7e"
format_stamp: "Formatted at 2026-06-18 22:30:07 on dist-test-slave-0956"
server_key: "d21f3092844ba29bbac61e0c99f86b05"
server_key_iv: "299a75de26f134e15214cd31c48a2637"
server_key_version: "encryptionkey@0"
I20260618 22:30:13.391494 26111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:13.405709 26111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:13.410332 26111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:13.410580 26111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:13.427371 26111 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:35805
I20260618 22:30:13.427393 26177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:35805 every 8 connection(s)
I20260618 22:30:13.428849 26111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:30:13.431007 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 26111
I20260618 22:30:13.431752 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:46011
--local_ip_for_outbound_sockets=127.10.242.193
--tserver_master_addrs=127.10.242.254:35805
--webserver_port=39771
--webserver_interface=127.10.242.193
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:30:13.435933 26178 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:30:13.439199 26178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e: Bootstrap starting.
I20260618 22:30:13.459254 26178 log.cc:824] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e: Log is configured to *not* fsync() on all Append() calls
I20260618 22:30:13.475768 26178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:30:13.476287 26178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e: Bootstrap complete.
I20260618 22:30:13.485258 26178 raft_consensus.cc:348] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffd12ba538c464b868b911063445d7e" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35805 } }
I20260618 22:30:13.486240 26178 raft_consensus.cc:729] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bffd12ba538c464b868b911063445d7e, State: Initialized, Role: FOLLOWER
I20260618 22:30:13.486866 26178 consensus_queue.cc:260] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffd12ba538c464b868b911063445d7e" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35805 } }
I20260618 22:30:13.487097 26178 raft_consensus.cc:388] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:30:13.487203 26178 raft_consensus.cc:482] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:30:13.487360 26178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 2 FOLLOWER]: Advancing to term 3
I20260618 22:30:13.491308 26178 raft_consensus.cc:504] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffd12ba538c464b868b911063445d7e" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35805 } }
I20260618 22:30:13.491802 26178 leader_election.cc:302] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bffd12ba538c464b868b911063445d7e; no voters:
I20260618 22:30:13.492445 26178 leader_election.cc:288] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260618 22:30:13.492638 26182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 3 FOLLOWER]: Leader election won for term 3
I20260618 22:30:13.493616 26182 raft_consensus.cc:686] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [term 3 LEADER]: Becoming Leader. State: Replica: bffd12ba538c464b868b911063445d7e, State: Running, Role: LEADER
I20260618 22:30:13.494436 26178 sys_catalog.cc:564] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:30:13.494376 26182 consensus_queue.cc:237] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffd12ba538c464b868b911063445d7e" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35805 } }
I20260618 22:30:13.502230 26184 sys_catalog.cc:455] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bffd12ba538c464b868b911063445d7e. Latest consensus state: current_term: 3 leader_uuid: "bffd12ba538c464b868b911063445d7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffd12ba538c464b868b911063445d7e" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35805 } } }
I20260618 22:30:13.502627 26184 sys_catalog.cc:458] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [sys.catalog]: This master's current role is: LEADER
I20260618 22:30:13.503170 26183 sys_catalog.cc:455] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "bffd12ba538c464b868b911063445d7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bffd12ba538c464b868b911063445d7e" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 35805 } } }
I20260618 22:30:13.503453 26183 sys_catalog.cc:458] T 00000000000000000000000000000000 P bffd12ba538c464b868b911063445d7e [sys.catalog]: This master's current role is: LEADER
I20260618 22:30:13.510309 26196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:30:13.518824 26196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:30:13.520373 26196 catalog_manager.cc:1259] Loaded cluster ID: bf63df4f06ed405ebb7fa01545ceaed8
I20260618 22:30:13.520505 26196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260618 22:30:13.522650 26196 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f75935419ad google::LogMessage::Fail() at ??:0
@ 0x7f759354488a google::LogMessage::SendToLog() at ??:0
@ 0x7f75935414de google::LogMessage::Flush() at ??:0
@ 0x7f7593542a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f75a8f74512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f75a8f71976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f75a8ff9610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f75a8ff9439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f759618d345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f759619091c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f7596190799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f759616beaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f75994456db start_thread at ??:0
@ 0x7f75923db61f clone at ??:0
W20260618 22:30:13.849756 26180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:13.850159 26180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:13.850312 26180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:13.858757 26180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:13.858880 26180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:13.859009 26180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:30:13.869015 26180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:46011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=39771
--tserver_master_addrs=127.10.242.254:35805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:13.870301 26180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:13.871608 26180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:13.884487 26221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:13.884528 26223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:13.884585 26220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:13.885319 26180 server_base.cc:1034] running on GCE node
I20260618 22:30:13.886008 26180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:13.886917 26180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:13.888134 26180 hybrid_clock.cc:648] HybridClock initialized: now 1781821813888083 us; error 35 us; skew 500 ppm
I20260618 22:30:13.888429 26180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:13 dist-test-slave-0956 krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821813, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:13.891639 26180 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:30:13.893069 26180 webserver.cc:460] Webserver started at http://127.10.242.193:39771/ using document root <none> and password file <none>
I20260618 22:30:13.893838 26180 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:13.893947 26180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:13.899520 26180 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260618 22:30:13.902360 26230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:13.903762 26180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260618 22:30:13.903923 26180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "87cf49fcc4194b5c9429af4a95fc9276"
format_stamp: "Formatted at 2026-06-18 22:30:08 on dist-test-slave-0956"
server_key: "ba3695311fb55c4c288cc1c013c54009"
server_key_iv: "a5576b23f636a677ca4993fc06de6ca5"
server_key_version: "encryptionkey@0"
I20260618 22:30:13.904925 26180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:13.923911 26180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:13.928411 26180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:13.928683 26180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:13.929711 26180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:30:13.931030 26180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:30:13.931128 26180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:13.931238 26180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:30:13.931310 26180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:13.962435 26180 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:46011
I20260618 22:30:13.962517 26342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:46011 every 8 connection(s)
I20260618 22:30:13.963878 26180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:30:13.967270 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 26180
I20260618 22:30:13.967873 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:33811
--local_ip_for_outbound_sockets=127.10.242.194
--tserver_master_addrs=127.10.242.254:35805
--webserver_port=43495
--webserver_interface=127.10.242.194
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:30:13.969012 26345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:13.966938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:57383)
0618 22:30:13.967762 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:13.968025 (+ 263us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":325,"threads_started":1}
W20260618 22:30:13.969664 26343 heartbeater.cc:644] Failed to heartbeat to 127.10.242.254:35805 (0 consecutive failures): Network error: Failed to ping master at 127.10.242.254:35805: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
I20260618 22:30:13.970552 26345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:13.970213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:39527)
0618 22:30:13.970358 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:13.970430 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260618 22:30:13.971385 26345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:13.971096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:37031)
0618 22:30:13.971210 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:13.971281 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260618 22:30:13.971603 26343 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260618 22:30:14.287781 26346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:14.288072 26346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:14.288172 26346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:14.295909 26346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:14.296034 26346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:14.296152 26346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:30:14.306139 26346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:33811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=43495
--tserver_master_addrs=127.10.242.254:35805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:14.307389 26346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:14.308761 26346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:14.321647 26351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:14.321578 26352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:14.321661 26354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:14.322440 26346 server_base.cc:1034] running on GCE node
I20260618 22:30:14.323069 26346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:14.323995 26346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:14.325178 26346 hybrid_clock.cc:648] HybridClock initialized: now 1781821814325119 us; error 54 us; skew 500 ppm
I20260618 22:30:14.325546 26346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:14 dist-test-slave-0956 krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821814, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:14.328724 26346 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:30:14.330225 26346 webserver.cc:460] Webserver started at http://127.10.242.194:43495/ using document root <none> and password file <none>
I20260618 22:30:14.330879 26346 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:14.330986 26346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:14.336198 26346 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260618 22:30:14.338843 26361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:14.340236 26346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:30:14.340391 26346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "cc33e2b9b6ed4040bc239bfeae5229a4"
format_stamp: "Formatted at 2026-06-18 22:30:08 on dist-test-slave-0956"
server_key: "39c155f6c1d3099b8e88140b6b5c04a5"
server_key_iv: "c6c2a8e75ba9ba4ba9504114c02fc366"
server_key_version: "encryptionkey@0"
I20260618 22:30:14.341435 26346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:14.366657 26346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:14.371034 26346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:14.371289 26346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:14.372277 26346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:30:14.373680 26346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:30:14.373780 26346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:14.373876 26346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:30:14.373950 26346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:14.404870 26346 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:33811
I20260618 22:30:14.404937 26473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:33811 every 8 connection(s)
I20260618 22:30:14.406345 26346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:30:14.409941 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 26346
I20260618 22:30:14.410679 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:42763
--local_ip_for_outbound_sockets=127.10.242.195
--tserver_master_addrs=127.10.242.254:35805
--webserver_port=38425
--webserver_interface=127.10.242.195
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:30:14.411754 26476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:14.409372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:59991)
0618 22:30:14.410278 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:14.410591 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":344,"threads_started":1}
W20260618 22:30:14.412452 26474 heartbeater.cc:644] Failed to heartbeat to 127.10.242.254:35805 (0 consecutive failures): Network error: Failed to ping master at 127.10.242.254:35805: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
I20260618 22:30:14.413328 26476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:14.412938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:50405)
0618 22:30:14.413119 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:14.413200 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260618 22:30:14.414223 26476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:14.413903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:60715)
0618 22:30:14.414031 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:14.414117 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260618 22:30:14.414429 26474 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260618 22:30:14.734031 26477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:14.734323 26477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:14.734457 26477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:14.742087 26477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:14.742221 26477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:14.742416 26477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:30:14.752476 26477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:42763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=38425
--tserver_master_addrs=127.10.242.254:35805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:14.753847 26477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:14.755172 26477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:14.767957 26483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:14.768127 26485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:14.768512 26482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:14.768668 26477 server_base.cc:1034] running on GCE node
I20260618 22:30:14.769407 26477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:14.770406 26477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:14.771631 26477 hybrid_clock.cc:648] HybridClock initialized: now 1781821814771583 us; error 35 us; skew 500 ppm
I20260618 22:30:14.771934 26477 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:14 dist-test-slave-0956 krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821814, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:14.775049 26477 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:30:14.776464 26477 webserver.cc:460] Webserver started at http://127.10.242.195:38425/ using document root <none> and password file <none>
I20260618 22:30:14.777158 26477 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:14.777261 26477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:14.782689 26477 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260618 22:30:14.785496 26492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:14.786911 26477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:30:14.787051 26477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "c84e78806f334957b5ecc9f59882f86b"
format_stamp: "Formatted at 2026-06-18 22:30:09 on dist-test-slave-0956"
server_key: "9c1afa46cc3be1eb2de26465ec16843d"
server_key_iv: "6567709e029397ceb1ed94afa45fbb39"
server_key_version: "encryptionkey@0"
I20260618 22:30:14.787986 26477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:14.808645 26477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:14.812894 26477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:14.813161 26477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:14.814113 26477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:30:14.815436 26477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:30:14.815593 26477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:14.815677 26477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:30:14.815737 26477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:14.845919 26477 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:42763
I20260618 22:30:14.845988 26604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:42763 every 8 connection(s)
I20260618 22:30:14.847841 26477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:30:14.853401 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 26477
I20260618 22:30:14.853957 26607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:14.851526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:36653)
0618 22:30:14.852596 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:14.852823 (+ 227us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":544,"threads_started":1}
W20260618 22:30:14.854673 26605 heartbeater.cc:644] Failed to heartbeat to 127.10.242.254:35805 (0 consecutive failures): Network error: Failed to ping master at 127.10.242.254:35805: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
I20260618 22:30:14.855508 26607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:14.855176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:47373)
0618 22:30:14.855321 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:14.855389 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260618 22:30:14.856621 26607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:14.856263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:41369)
0618 22:30:14.856403 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:14.856511 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260618 22:30:14.856874 26605 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260618 22:30:14.973044 26609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:14.972170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:40971)
0618 22:30:14.972676 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:14.972814 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":226,"threads_started":1}
I20260618 22:30:15.416030 26610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:15.415099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:47065)
0618 22:30:15.415633 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:15.415782 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":259,"threads_started":1}
I20260618 22:30:15.858634 26611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:15.857650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:56367)
0618 22:30:15.858225 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:15.858359 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":297,"threads_started":1}
I20260618 22:30:15.974747 26612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:15.973912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:35977)
0618 22:30:15.974449 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:15.974554 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":269,"threads_started":1}
I20260618 22:30:16.417796 26613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:16.416976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:36881)
0618 22:30:16.417528 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:16.417622 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":262,"threads_started":1}
I20260618 22:30:16.860658 26614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:16.859756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:56557)
0618 22:30:16.860320 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:16.860463 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":285,"threads_started":1}
I20260618 22:30:16.976552 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:16.975734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:57607)
0618 22:30:16.976241 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:16.976327 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":295,"threads_started":1}
I20260618 22:30:17.419404 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:17.418686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:57665)
0618 22:30:17.419132 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:17.419222 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":246,"threads_started":1}
I20260618 22:30:17.862527 26617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:17.861687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:45161)
0618 22:30:17.862242 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:17.862331 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":330,"threads_started":1}
I20260618 22:30:17.978215 26618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:17.977474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:47787)
0618 22:30:17.977923 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:17.978006 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":225,"threads_started":1}
I20260618 22:30:18.421057 26619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:18.420275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:46453)
0618 22:30:18.420783 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:18.420865 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":279,"threads_started":1}
I20260618 22:30:18.864142 26620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:18.863445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:43323)
0618 22:30:18.863866 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:18.863949 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":204,"threads_started":1}
I20260618 22:30:18.980073 26621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:18.979300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:47043)
0618 22:30:18.979809 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:18.979896 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":289,"threads_started":1}
I20260618 22:30:19.422896 26622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:19.422107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:34917)
0618 22:30:19.422628 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:19.422727 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":234,"threads_started":1}
I20260618 22:30:19.865800 26623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:19.865036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:52059)
0618 22:30:19.865541 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:19.865629 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":284,"threads_started":1}
I20260618 22:30:19.981767 26624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:19.980991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:43249)
0618 22:30:19.981503 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:19.981600 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":277,"threads_started":1}
I20260618 22:30:20.424713 26625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:20.423942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:39483)
0618 22:30:20.424435 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:20.424539 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":257,"threads_started":1}
I20260618 22:30:20.867406 26626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:20.866617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:48439)
0618 22:30:20.867130 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:20.867223 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":250,"threads_started":1}
I20260618 22:30:20.983564 26627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:20.982714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:44573)
0618 22:30:20.983245 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:20.983338 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":225,"threads_started":1}
I20260618 22:30:21.426573 26628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:21.425742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:38773)
0618 22:30:21.426265 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:21.426357 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":209,"threads_started":1}
I20260618 22:30:21.869168 26629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:21.868323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:40995)
0618 22:30:21.868837 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:21.868923 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":218,"threads_started":1}
I20260618 22:30:21.985370 26630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:21.984560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:57677)
0618 22:30:21.985054 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:21.985187 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":245,"threads_started":1}
I20260618 22:30:22.428256 26631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:22.427514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:36399)
0618 22:30:22.427974 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:22.428065 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":236,"threads_started":1}
I20260618 22:30:22.870939 26632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:22.870163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:37853)
0618 22:30:22.870668 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:22.870771 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":251,"threads_started":1}
I20260618 22:30:22.987054 26633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:22.986319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:39377)
0618 22:30:22.986802 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:22.986888 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":212,"threads_started":1}
I20260618 22:30:23.429988 26634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:23.429238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:46907)
0618 22:30:23.429720 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:23.429809 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":215,"threads_started":1}
I20260618 22:30:23.872637 26635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:23.871877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:45365)
0618 22:30:23.872376 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:23.872465 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":244,"threads_started":1}
I20260618 22:30:23.988651 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:23.987942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:37701)
0618 22:30:23.988411 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:23.988494 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":283,"threads_started":1}
I20260618 22:30:24.431710 26637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:24.430967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:59195)
0618 22:30:24.431446 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:24.431536 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":258,"threads_started":1}
I20260618 22:30:24.874325 26638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:24.873578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:60919)
0618 22:30:24.874074 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:24.874162 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":267,"threads_started":1}
I20260618 22:30:24.990298 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:24.989575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:34297)
0618 22:30:24.990027 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:24.990123 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":220,"threads_started":1}
I20260618 22:30:25.433552 26640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:25.432732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:34065)
0618 22:30:25.433235 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:25.433343 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":225,"threads_started":1}
I20260618 22:30:25.876113 26641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:25.875314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:55945)
0618 22:30:25.875803 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:25.875915 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":234,"threads_started":1}
I20260618 22:30:25.992108 26642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:25.991350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:58435)
0618 22:30:25.991864 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:25.991956 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":259,"threads_started":1}
I20260618 22:30:26.435344 26643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:26.434618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:38813)
0618 22:30:26.435083 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:26.435176 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":237,"threads_started":1}
I20260618 22:30:26.877910 26644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:26.877136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:47363)
0618 22:30:26.877604 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:26.877696 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":242,"threads_started":1}
I20260618 22:30:26.993866 26645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:26.993055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:35549)
0618 22:30:26.993569 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:26.993672 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":220,"threads_started":1}
I20260618 22:30:27.437031 26646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:27.436262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:47853)
0618 22:30:27.436708 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:27.436807 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":215,"threads_started":1}
I20260618 22:30:27.879670 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:27.878908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:34133)
0618 22:30:27.879379 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:27.879497 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":239,"threads_started":1}
I20260618 22:30:27.995630 26648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:27.994796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:40717)
0618 22:30:27.995314 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:27.995419 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":260,"threads_started":1}
I20260618 22:30:28.438745 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:28.437938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:51109)
0618 22:30:28.438445 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:28.438543 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":240,"threads_started":1}
I20260618 22:30:28.881475 26650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:28.880684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:53385)
0618 22:30:28.881210 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:28.881300 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":249,"threads_started":1}
I20260618 22:30:28.997373 26651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:28.996645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.193:57509)
0618 22:30:28.997070 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:28.997197 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":216,"threads_started":1}
I20260618 22:30:29.440896 26652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:29.440070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.194:43647)
0618 22:30:29.440554 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:29.440685 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":239,"threads_started":1}
I20260618 22:30:29.860034 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 26180
I20260618 22:30:29.884570 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:29.883624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:35805 (local address 127.10.242.195:56655)
0618 22:30:29.884181 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:29.884334 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":261,"threads_started":1}
I20260618 22:30:29.892205 26653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:29.861382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44452 (local address 127.10.242.193:46011)
0618 22:30:29.862019 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:29.862038 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:29.862091 (+ 53us) server_negotiation.cc:407] Connection header received
0618 22:30:29.864078 (+ 1987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:29.864089 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:29.864539 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:29.864845 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:29.866371 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:29.867922 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:29.869861 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:29.871063 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:29.871252 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:29.883736 (+ 12484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:29.883750 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:29.883777 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:29.883850 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:29.888215 (+ 4365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:29.889299 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:29.889308 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:29.889316 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:29.889505 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:29.889934 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:29.889940 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:29.889943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:29.890554 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:29.890860 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:29.891385 (+ 525us) server_negotiation.cc:299] Negotiation successful
0618 22:30:29.891925 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":305,"threads_started":1}
I20260618 22:30:29.894086 26338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:29.990517 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 26180
I20260618 22:30:30.004364 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 26346
I20260618 22:30:30.023833 26656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:30.005488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50992 (local address 127.10.242.194:33811)
0618 22:30:30.006064 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:30.006078 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:30.006118 (+ 40us) server_negotiation.cc:407] Connection header received
0618 22:30:30.007762 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:30.007774 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:30.008255 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:30.008609 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:30.010242 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:30.011613 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:30.012993 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:30.013827 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:30.013998 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:30.016734 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:30.016742 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:30.016761 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:30.016824 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:30.020156 (+ 3332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:30.021012 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:30.021021 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:30.021028 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:30.021342 (+ 314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:30.021801 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:30.021805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:30.021808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:30.022413 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:30.022657 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:30.023108 (+ 451us) server_negotiation.cc:299] Negotiation successful
0618 22:30:30.023540 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":262,"threads_started":1}
I20260618 22:30:30.025314 26469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:30.109694 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 26346
I20260618 22:30:30.124015 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 26477
I20260618 22:30:30.142282 26658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:30.125013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38896 (local address 127.10.242.195:42763)
0618 22:30:30.125653 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:30.125666 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:30.125708 (+ 42us) server_negotiation.cc:407] Connection header received
0618 22:30:30.127352 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:30.127362 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:30.127793 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:30.128170 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:30.129683 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:30.130943 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:30.132270 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:30.132971 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:30.133167 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:30.135675 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:30.135683 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:30.135699 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:30.135740 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:30.138866 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:30.139592 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:30.139599 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:30.139606 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:30.139773 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:30.140137 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:30.140140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:30.140142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:30.140673 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:30.140968 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:30.141547 (+ 579us) server_negotiation.cc:299] Negotiation successful
0618 22:30:30.141988 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":328,"threads_started":1}
I20260618 22:30:30.143680 26600 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:30.228534 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 26477
2026-06-18T22:30:30Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24250 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:30:30 dist-test-slave-0956 krb5kdc[26662](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:30:30 dist-test-slave-0956 krb5kdc[26662](info): set up 2 sockets
Jun 18 22:30:30 dist-test-slave-0956 krb5kdc[26662](info): commencing operation
krb5kdc: starting...
W20260618 22:30:31.257568 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:30:31 dist-test-slave-0956 krb5kdc[26662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:30:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:30:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:30:31.492350 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:40485
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:40485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:30:31.821943 26675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:31.822256 26675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:31.822378 26675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:31.829986 26675 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260618 22:30:31.830098 26675 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260618 22:30:31.830142 26675 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260618 22:30:31.830214 26675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:30:31.830282 26675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:31.830375 26675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:31.830433 26675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:30:31.830507 26675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:30:31.840816 26675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:40485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:40485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:31.842206 26675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:31.843609 26675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:31.856107 26680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:31.856148 26681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:31.856268 26683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:31.857290 26675 server_base.cc:1034] running on GCE node
I20260618 22:30:31.858129 26675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:31.859685 26675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:31.860980 26675 hybrid_clock.cc:648] HybridClock initialized: now 1781821831860894 us; error 60 us; skew 500 ppm
I20260618 22:30:31.861367 26675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:31 dist-test-slave-0956 krb5kdc[26662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821831, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:31.864621 26675 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:30:31.866218 26675 webserver.cc:460] Webserver started at http://127.10.242.254:35479/ using document root <none> and password file <none>
I20260618 22:30:31.866914 26675 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:31.867004 26675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:31.867303 26675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:30:31.870428 26675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "b8c6ff6457674018b0ccbf5a46d88e4b"
format_stamp: "Formatted at 2026-06-18 22:30:31 on dist-test-slave-0956"
server_key: "7b1f9a1ae2a5b094ee5efbe16fd18959"
server_key_iv: "084e16f71dcf488fb9c884ab19e471cb"
server_key_version: "encryptionkey@0"
I20260618 22:30:31.871234 26675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "b8c6ff6457674018b0ccbf5a46d88e4b"
format_stamp: "Formatted at 2026-06-18 22:30:31 on dist-test-slave-0956"
server_key: "7b1f9a1ae2a5b094ee5efbe16fd18959"
server_key_iv: "084e16f71dcf488fb9c884ab19e471cb"
server_key_version: "encryptionkey@0"
I20260618 22:30:31.876809 26675 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260618 22:30:31.880564 26690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:31.882071 26675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:30:31.882238 26675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "b8c6ff6457674018b0ccbf5a46d88e4b"
format_stamp: "Formatted at 2026-06-18 22:30:31 on dist-test-slave-0956"
server_key: "7b1f9a1ae2a5b094ee5efbe16fd18959"
server_key_iv: "084e16f71dcf488fb9c884ab19e471cb"
server_key_version: "encryptionkey@0"
I20260618 22:30:31.882452 26675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:31.898245 26675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:31.902563 26675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:31.902817 26675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:31.920531 26675 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:40485
I20260618 22:30:31.920542 26741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:40485 every 8 connection(s)
I20260618 22:30:31.921933 26675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:30:31.924482 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 26675
I20260618 22:30:31.924714 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:30:31.925609 11211 external_mini_cluster.cc:1428] Setting key 5135b030c88f9abec474d1cb45fba373
I20260618 22:30:31.926302 26742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 18 22:30:31 dist-test-slave-0956 krb5kdc[26662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:30:31.941562 26742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: Bootstrap starting.
I20260618 22:30:31.945240 26742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: Neither blocks nor log segments found. Creating new log.
I20260618 22:30:31.946595 26742 log.cc:824] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:30:31.949312 26745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:31.927708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53382 (local address 127.10.242.254:40485)
0618 22:30:31.928565 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:31.928581 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:31.928633 (+ 52us) server_negotiation.cc:407] Connection header received
0618 22:30:31.930429 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:31.930458 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:31.931011 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:31.931476 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:31.932670 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:31.934819 (+ 2149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:31.936240 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:31.937387 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:31.937593 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:31.941895 (+ 4302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:31.941904 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:31.941920 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:31.941959 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:31.944851 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:31.945848 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:31.945855 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:31.945862 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:31.946038 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:31.946492 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:31.946495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:31.946497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:31.947074 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:31.947371 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:31.947853 (+ 482us) server_negotiation.cc:299] Negotiation successful
0618 22:30:31.948269 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":357,"threads_started":1}
I20260618 22:30:31.949890 26742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: No bootstrap required, opened a new log
I20260618 22:30:31.958285 26742 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40485 } }
I20260618 22:30:31.958798 26742 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:30:31.958894 26742 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c6ff6457674018b0ccbf5a46d88e4b, State: Initialized, Role: FOLLOWER
I20260618 22:30:31.959471 26742 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40485 } }
I20260618 22:30:31.959674 26742 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:30:31.959769 26742 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:30:31.959892 26742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:30:31.962057 26742 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40485 } }
I20260618 22:30:31.962500 26742 leader_election.cc:302] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8c6ff6457674018b0ccbf5a46d88e4b; no voters:
I20260618 22:30:31.963022 26742 leader_election.cc:288] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:30:31.963142 26747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:30:31.964213 26747 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 1 LEADER]: Becoming Leader. State: Replica: b8c6ff6457674018b0ccbf5a46d88e4b, State: Running, Role: LEADER
I20260618 22:30:31.964705 26747 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40485 } }
I20260618 22:30:31.965170 26742 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:30:31.967257 26749 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8c6ff6457674018b0ccbf5a46d88e4b. Latest consensus state: current_term: 1 leader_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40485 } } }
I20260618 22:30:31.967648 26749 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [sys.catalog]: This master's current role is: LEADER
I20260618 22:30:31.969231 26753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:30:31.969123 26748 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40485 } } }
I20260618 22:30:31.969426 26748 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [sys.catalog]: This master's current role is: LEADER
I20260618 22:30:31.974514 26753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:30:31.983011 26753 catalog_manager.cc:1347] Generated new cluster ID: 8e89f0d1bb6e47c68173ab68c89b3fe1
I20260618 22:30:31.983132 26753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:30:32.002003 26753 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:30:32.003383 26753 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:30:32.013466 26753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: Generated new TSK 0
I20260618 22:30:32.014308 26753 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
I20260618 22:30:32.084081 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:40485
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:30:32.455272 26768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:32.455547 26768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:32.455672 26768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:32.463101 26768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:32.463209 26768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:32.463356 26768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:30:32.473353 26768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:40485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:32.474613 26768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:32.475916 26768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:32.488445 26773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:32.488485 26774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:32.488693 26776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:32.489599 26768 server_base.cc:1034] running on GCE node
I20260618 22:30:32.490226 26768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:32.491186 26768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:32.492384 26768 hybrid_clock.cc:648] HybridClock initialized: now 1781821832492334 us; error 52 us; skew 500 ppm
I20260618 22:30:32.492681 26768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:32 dist-test-slave-0956 krb5kdc[26662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821832, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:32.495612 26768 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:30:32.496984 26768 webserver.cc:460] Webserver started at http://127.10.242.193:44709/ using document root <none> and password file <none>
I20260618 22:30:32.497679 26768 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:32.497782 26768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:32.498056 26768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:30:32.500851 26768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "7d8b95883e6047f09c698c5f55f90977"
format_stamp: "Formatted at 2026-06-18 22:30:32 on dist-test-slave-0956"
server_key: "32fb911939ca83d16524223987e420b1"
server_key_iv: "48c372ad792a680226d076523c6251e9"
server_key_version: "encryptionkey@0"
I20260618 22:30:32.501631 26768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "7d8b95883e6047f09c698c5f55f90977"
format_stamp: "Formatted at 2026-06-18 22:30:32 on dist-test-slave-0956"
server_key: "32fb911939ca83d16524223987e420b1"
server_key_iv: "48c372ad792a680226d076523c6251e9"
server_key_version: "encryptionkey@0"
I20260618 22:30:32.506601 26768 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:30:32.509923 26783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:32.511185 26768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:30:32.511328 26768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "7d8b95883e6047f09c698c5f55f90977"
format_stamp: "Formatted at 2026-06-18 22:30:32 on dist-test-slave-0956"
server_key: "32fb911939ca83d16524223987e420b1"
server_key_iv: "48c372ad792a680226d076523c6251e9"
server_key_version: "encryptionkey@0"
I20260618 22:30:32.511492 26768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:32.530594 26768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:32.534536 26768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:32.534811 26768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:32.535730 26768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:30:32.536998 26768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:30:32.537070 26768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:32.537223 26768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:30:32.537279 26768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:32.567296 26768 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:39701
I20260618 22:30:32.567341 26895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:39701 every 8 connection(s)
I20260618 22:30:32.568557 26768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:30:32.570075 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 26768
I20260618 22:30:32.570272 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:30:32.570884 11211 external_mini_cluster.cc:1428] Setting key 18d1bb3313e0a9fb4f0e0813adce0a9b
Jun 18 22:30:32 dist-test-slave-0956 krb5kdc[26662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821832, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:30:32.588255 26898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:32.571147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:59567 (local address 127.10.242.254:40485)
0618 22:30:32.571599 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:32.571616 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:32.572687 (+ 1071us) server_negotiation.cc:407] Connection header received
0618 22:30:32.574570 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:32.574576 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:32.574676 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:32.574766 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:32.576260 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:32.576958 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:32.578504 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:32.578903 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:32.579024 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:32.582575 (+ 3551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:32.582580 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:32.582583 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:32.582618 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:32.585031 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:32.585887 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:32.585893 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:32.585896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:32.586016 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:32.586459 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:32.586464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:32.586467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:32.586822 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:32.586964 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:32.587664 (+ 700us) server_negotiation.cc:299] Negotiation successful
0618 22:30:32.587889 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":206,"threads_started":1}
I20260618 22:30:32.589143 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:32.571765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:59567)
0618 22:30:32.572533 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:32.572583 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:32.574311 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:32.575012 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:32.575020 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:32.575524 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:32.576086 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:32.576114 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:32.577134 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:32.577138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:32.578364 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:32.578376 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:32.579126 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:32.579131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:32.579338 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:32.579972 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:32.579999 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:32.582336 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:32.585205 (+ 2869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:32.585214 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:32.585231 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:32.585756 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:32.586109 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:32.586113 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:32.586117 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:32.586332 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:32.586946 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:32.586955 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:32.587287 (+ 332us) client_negotiation.cc:769] Sending connection context
0618 22:30:32.587531 (+ 244us) client_negotiation.cc:240] Negotiation successful
0618 22:30:32.587971 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":329,"threads_started":1}
I20260618 22:30:32.590624 26896 heartbeater.cc:347] Connected to a master server at 127.10.242.254:40485
I20260618 22:30:32.590900 26896 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.10.242.194@KRBTEST.COM; defaulting to no policy
I20260618 22:30:32.591619 26896 heartbeater.cc:511] Master 127.10.242.254:40485 requested a full tablet report, sending...
I20260618 22:30:32.593732 26707 ts_manager.cc:194] Registered new tserver with Master: 7d8b95883e6047f09c698c5f55f90977 (127.10.242.193:39701)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.194@KRBTEST.COM" created.
I20260618 22:30:32.595654 26707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:59567
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
I20260618 22:30:32.620256 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:40485
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:30:32.938982 26902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:32.939307 26902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:32.939450 26902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:32.947002 26902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:32.947161 26902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:32.947333 26902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:30:32.957806 26902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:40485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:32.959138 26902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:32.960482 26902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:32.973860 26908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:32.973891 26910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:32.973944 26907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:32.975432 26902 server_base.cc:1034] running on GCE node
I20260618 22:30:32.976052 26902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:32.977001 26902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:32.978215 26902 hybrid_clock.cc:648] HybridClock initialized: now 1781821832978158 us; error 43 us; skew 500 ppm
I20260618 22:30:32.978557 26902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:32 dist-test-slave-0956 krb5kdc[26662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821832, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:32.981851 26902 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:30:32.983238 26902 webserver.cc:460] Webserver started at http://127.10.242.194:39637/ using document root <none> and password file <none>
I20260618 22:30:32.983904 26902 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:32.984004 26902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:32.984244 26902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:30:32.987308 26902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "dc19bc4ce9844a68a5401b9ee888f302"
format_stamp: "Formatted at 2026-06-18 22:30:32 on dist-test-slave-0956"
server_key: "5501cc29c30685023820244dd6eb1af2"
server_key_iv: "34ca8ff20a138913d0d6084b8faa89d7"
server_key_version: "encryptionkey@0"
I20260618 22:30:32.988022 26902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "dc19bc4ce9844a68a5401b9ee888f302"
format_stamp: "Formatted at 2026-06-18 22:30:32 on dist-test-slave-0956"
server_key: "5501cc29c30685023820244dd6eb1af2"
server_key_iv: "34ca8ff20a138913d0d6084b8faa89d7"
server_key_version: "encryptionkey@0"
I20260618 22:30:32.993373 26902 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260618 22:30:32.997001 26917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:32.998459 26902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:30:32.998608 26902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "dc19bc4ce9844a68a5401b9ee888f302"
format_stamp: "Formatted at 2026-06-18 22:30:32 on dist-test-slave-0956"
server_key: "5501cc29c30685023820244dd6eb1af2"
server_key_iv: "34ca8ff20a138913d0d6084b8faa89d7"
server_key_version: "encryptionkey@0"
I20260618 22:30:32.998807 26902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:33.019484 26902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:33.023890 26902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:33.024176 26902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:33.025202 26902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:30:33.026549 26902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:30:33.026626 26902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:33.026726 26902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:30:33.026777 26902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:33.057878 26902 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:43839
I20260618 22:30:33.058034 27029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:43839 every 8 connection(s)
I20260618 22:30:33.059237 26902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:30:33.061749 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 26902
I20260618 22:30:33.061936 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:30:33.062484 11211 external_mini_cluster.cc:1428] Setting key 7f2be603e92caf28120a0e67fcc130d8
Jun 18 22:30:33 dist-test-slave-0956 krb5kdc[26662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821832, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:30:33.080123 26898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:33.061921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:59531 (local address 127.10.242.254:40485)
0618 22:30:33.062097 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:33.062102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:33.063636 (+ 1534us) server_negotiation.cc:407] Connection header received
0618 22:30:33.065594 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:33.065599 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:33.065683 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:33.065766 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:33.067324 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.068067 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.069806 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.070408 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.070539 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:33.074221 (+ 3682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:33.074226 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:33.074230 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:33.074266 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:33.076908 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:33.077825 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:33.077829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:33.077831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:33.077973 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:33.078428 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:33.078432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:33.078434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:33.078713 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:33.078823 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:33.079660 (+ 837us) server_negotiation.cc:299] Negotiation successful
0618 22:30:33.079862 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260618 22:30:33.081559 27032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:33.062557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:59531)
0618 22:30:33.063450 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:33.063522 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:33.065311 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:33.066038 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:33.066047 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:33.066570 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:33.067139 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:33.067157 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.068214 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.068218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:33.069640 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:33.069653 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.070631 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.070638 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:33.070882 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:33.071524 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:33.071551 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:33.073975 (+ 2424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:33.077043 (+ 3068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:33.077052 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:33.077069 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:33.077698 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:33.078072 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:33.078077 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:33.078080 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:33.078299 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:33.078824 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:33.078832 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:33.079239 (+ 407us) client_negotiation.cc:769] Sending connection context
0618 22:30:33.079525 (+ 286us) client_negotiation.cc:240] Negotiation successful
0618 22:30:33.080091 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":377,"threads_started":1}
I20260618 22:30:33.083280 27030 heartbeater.cc:347] Connected to a master server at 127.10.242.254:40485
I20260618 22:30:33.083650 27030 heartbeater.cc:464] Registering TS with master...
I20260618 22:30:33.084444 27030 heartbeater.cc:511] Master 127.10.242.254:40485 requested a full tablet report, sending...
I20260618 22:30:33.086134 26707 ts_manager.cc:194] Registered new tserver with Master: dc19bc4ce9844a68a5401b9ee888f302 (127.10.242.194:43839)
WARNING: no policy specified for kudu/127.10.242.195@KRBTEST.COM; defaulting to no policy
I20260618 22:30:33.087240 26707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:59531
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
I20260618 22:30:33.116458 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:40485
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:30:33.440142 27035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:33.440448 27035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:33.440572 27035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:33.448074 27035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:33.448230 27035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:33.448395 27035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:30:33.458683 27035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:40485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:33.460156 27035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:33.461719 27035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:33.475167 27040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:33.475134 27041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:33.475562 27043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:33.476070 27035 server_base.cc:1034] running on GCE node
I20260618 22:30:33.476727 27035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:33.477739 27035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:33.479005 27035 hybrid_clock.cc:648] HybridClock initialized: now 1781821833478936 us; error 53 us; skew 500 ppm
I20260618 22:30:33.479349 27035 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:33 dist-test-slave-0956 krb5kdc[26662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821833, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:33.482818 27035 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:30:33.484503 27035 webserver.cc:460] Webserver started at http://127.10.242.195:41263/ using document root <none> and password file <none>
I20260618 22:30:33.485411 27035 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:33.485549 27035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:33.485885 27035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:30:33.489005 27035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "d0f3eb64d664474cbf8679f78a965f2b"
format_stamp: "Formatted at 2026-06-18 22:30:33 on dist-test-slave-0956"
server_key: "90dd29a27cdf9e8e36d4506069f9590a"
server_key_iv: "b4aef2d90f23d9b1e91055358e7a127c"
server_key_version: "encryptionkey@0"
I20260618 22:30:33.489863 27035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "d0f3eb64d664474cbf8679f78a965f2b"
format_stamp: "Formatted at 2026-06-18 22:30:33 on dist-test-slave-0956"
server_key: "90dd29a27cdf9e8e36d4506069f9590a"
server_key_iv: "b4aef2d90f23d9b1e91055358e7a127c"
server_key_version: "encryptionkey@0"
I20260618 22:30:33.495693 27035 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:30:33.499657 27050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:33.501261 27035 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:30:33.501430 27035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "d0f3eb64d664474cbf8679f78a965f2b"
format_stamp: "Formatted at 2026-06-18 22:30:33 on dist-test-slave-0956"
server_key: "90dd29a27cdf9e8e36d4506069f9590a"
server_key_iv: "b4aef2d90f23d9b1e91055358e7a127c"
server_key_version: "encryptionkey@0"
I20260618 22:30:33.501628 27035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:33.516605 27035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:33.521405 27035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:33.521716 27035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:33.522735 27035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:30:33.524195 27035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:30:33.524276 27035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:33.524397 27035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:30:33.524458 27035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:33.557597 27035 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:43709
I20260618 22:30:33.557652 27162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:43709 every 8 connection(s)
I20260618 22:30:33.558897 27035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:30:33.568806 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 27035
I20260618 22:30:33.568996 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:30:33.569583 11211 external_mini_cluster.cc:1428] Setting key baf7038856f5b4a41cfe7a4a43d37320
Jun 18 22:30:33 dist-test-slave-0956 krb5kdc[26662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821833, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:30:33.580976 26898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:33.561925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:46635 (local address 127.10.242.254:40485)
0618 22:30:33.562106 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:33.562112 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:33.563365 (+ 1253us) server_negotiation.cc:407] Connection header received
0618 22:30:33.565275 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:33.565280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:33.565368 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:33.565463 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:33.567279 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.568032 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.570348 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.570969 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.571101 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:33.574915 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:33.574920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:33.574923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:33.575001 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:33.577675 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:33.578508 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:33.578514 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:33.578517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:33.578669 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:33.579142 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:33.579147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:33.579149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:33.579462 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:33.579602 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:33.580501 (+ 899us) server_negotiation.cc:299] Negotiation successful
0618 22:30:33.580711 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:30:33.582100 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:33.562349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:46635)
0618 22:30:33.563175 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:33.563216 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:33.564918 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:33.565761 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:33.565770 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:33.566387 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:33.567077 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:33.567099 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.568191 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.568195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:33.570178 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:33.570194 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.571211 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.571219 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:33.571533 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:33.572317 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:33.572342 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:33.574690 (+ 2348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:33.577842 (+ 3152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:33.577849 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:33.577865 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:33.578344 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:33.578788 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:33.578792 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:33.578794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:33.579002 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:33.579645 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:33.579652 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:33.580089 (+ 437us) client_negotiation.cc:769] Sending connection context
0618 22:30:33.580356 (+ 267us) client_negotiation.cc:240] Negotiation successful
0618 22:30:33.580914 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":310,"threads_started":1}
I20260618 22:30:33.583760 27163 heartbeater.cc:347] Connected to a master server at 127.10.242.254:40485
I20260618 22:30:33.584165 27163 heartbeater.cc:464] Registering TS with master...
I20260618 22:30:33.585067 27163 heartbeater.cc:511] Master 127.10.242.254:40485 requested a full tablet report, sending...
I20260618 22:30:33.586654 26707 ts_manager.cc:194] Registered new tserver with Master: d0f3eb64d664474cbf8679f78a965f2b (127.10.242.195:43709)
I20260618 22:30:33.587534 26707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:46635
I20260618 22:30:33.598455 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:30:33.598757 26896 heartbeater.cc:503] Master 127.10.242.254:40485 was elected leader, sending a full tablet report...
I20260618 22:30:33.616676 26898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:33.603101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53388 (local address 127.10.242.254:40485)
0618 22:30:33.603290 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:33.603298 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:33.603698 (+ 400us) server_negotiation.cc:407] Connection header received
0618 22:30:33.603852 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:33.603856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:33.603922 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:33.603996 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:33.605716 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.606565 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.608036 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.608485 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.608589 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:33.611120 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:33.611126 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:33.611129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:33.611168 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:33.613841 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:33.614569 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:33.614574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:33.614576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:33.614734 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:33.615214 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:33.615218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:33.615220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:33.615609 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:33.615812 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:33.616153 (+ 341us) server_negotiation.cc:299] Negotiation successful
0618 22:30:33.616340 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:30:33.620903 26707 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:30:33.627660 26707 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:30:33.650416 27176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:33.640111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52152 (local address 127.10.242.195:43709)
0618 22:30:33.640864 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:33.640872 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:33.640906 (+ 34us) server_negotiation.cc:407] Connection header received
0618 22:30:33.641044 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:33.641049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:33.641286 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:33.641457 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:33.642167 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.643834 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.646919 (+ 3085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.648578 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.648765 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:33.648970 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:33.649793 (+ 823us) server_negotiation.cc:299] Negotiation successful
0618 22:30:33.650005 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":425,"threads_started":1}
I20260618 22:30:33.651336 27175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:33.639818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:43709 (local address 127.0.0.1:52152)
0618 22:30:33.640596 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:33.640646 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:33.640816 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:33.641480 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:33.641487 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:33.641517 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:33.641936 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:33.641954 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.644090 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.644097 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:33.646682 (+ 2585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:33.646702 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.649200 (+ 2498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.649208 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:33.649475 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:33.649532 (+ 57us) client_negotiation.cc:769] Sending connection context
0618 22:30:33.650677 (+ 1145us) client_negotiation.cc:240] Negotiation successful
0618 22:30:33.650933 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":938,"threads_started":1}
I20260618 22:30:33.657634 27098 tablet_service.cc:1467] Processing CreateTablet for tablet e002cb45759f4035a812ea083402a79c (DEFAULT_TABLE table=test-table [id=ae6cbfbd1a9241a2b5c119d3858099c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:30:33.659575 27098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e002cb45759f4035a812ea083402a79c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:30:33.660681 27175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:33.642234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39701 (local address 127.0.0.1:43512)
0618 22:30:33.651697 (+ 9463us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:33.651719 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:33.651835 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:33.653314 (+ 1479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:33.653319 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:33.653356 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:33.653652 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:33.653665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.655988 (+ 2323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.655993 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:33.658079 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:33.658099 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.659895 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.659902 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:33.660094 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:33.660131 (+ 37us) client_negotiation.cc:769] Sending connection context
0618 22:30:33.660233 (+ 102us) client_negotiation.cc:240] Negotiation successful
0618 22:30:33.660387 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9342,"thread_start_us":10878,"threads_started":1}
I20260618 22:30:33.660770 27180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:33.642746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43512 (local address 127.10.242.193:39701)
0618 22:30:33.652545 (+ 9799us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:33.652554 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:33.652589 (+ 35us) server_negotiation.cc:407] Connection header received
0618 22:30:33.652698 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:33.652707 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:33.653013 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:33.653858 (+ 845us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:33.654133 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.655812 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.658272 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.659677 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.659914 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:33.660086 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:33.660287 (+ 201us) server_negotiation.cc:299] Negotiation successful
0618 22:30:33.660452 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9630,"thread_start_us":9471,"threads_started":1}
I20260618 22:30:33.661916 27177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:33.641458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:43839 (local address 127.0.0.1:36258)
0618 22:30:33.642103 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:33.642119 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:33.642215 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:33.654083 (+ 11868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:33.654088 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:33.654116 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:33.654420 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:33.654431 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.657842 (+ 3411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.657848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:33.659370 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:33.659384 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.661194 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.661199 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:33.661362 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:33.661395 (+ 33us) client_negotiation.cc:769] Sending connection context
0618 22:30:33.661465 (+ 70us) client_negotiation.cc:240] Negotiation successful
0618 22:30:33.661613 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":368,"threads_started":1}
I20260618 22:30:33.662788 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:33.641728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36258 (local address 127.10.242.194:43839)
0618 22:30:33.652949 (+ 11221us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:33.652957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:33.652984 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:30:33.653071 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:33.653110 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:33.653862 (+ 752us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:33.654129 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:33.656086 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.657612 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.659548 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.660946 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.662055 (+ 1109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:33.662233 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:33.662304 (+ 71us) server_negotiation.cc:299] Negotiation successful
0618 22:30:33.662465 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11060,"thread_start_us":10925,"threads_started":1}
I20260618 22:30:33.665228 26831 tablet_service.cc:1467] Processing CreateTablet for tablet e002cb45759f4035a812ea083402a79c (DEFAULT_TABLE table=test-table [id=ae6cbfbd1a9241a2b5c119d3858099c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:30:33.666936 26831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e002cb45759f4035a812ea083402a79c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:30:33.666756 26965 tablet_service.cc:1467] Processing CreateTablet for tablet e002cb45759f4035a812ea083402a79c (DEFAULT_TABLE table=test-table [id=ae6cbfbd1a9241a2b5c119d3858099c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:30:33.668164 26965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e002cb45759f4035a812ea083402a79c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:30:33.672313 27181 tablet_bootstrap.cc:492] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b: Bootstrap starting.
I20260618 22:30:33.675484 27181 tablet_bootstrap.cc:654] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b: Neither blocks nor log segments found. Creating new log.
I20260618 22:30:33.676491 27181 log.cc:824] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:30:33.679100 27181 tablet_bootstrap.cc:492] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b: No bootstrap required, opened a new log
I20260618 22:30:33.679410 27181 ts_tablet_manager.cc:1397] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260618 22:30:33.682235 27183 tablet_bootstrap.cc:492] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977: Bootstrap starting.
I20260618 22:30:33.685971 27184 tablet_bootstrap.cc:492] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302: Bootstrap starting.
I20260618 22:30:33.686324 27183 tablet_bootstrap.cc:654] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977: Neither blocks nor log segments found. Creating new log.
I20260618 22:30:33.688083 27183 log.cc:824] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977: Log is configured to *not* fsync() on all Append() calls
I20260618 22:30:33.690107 27184 tablet_bootstrap.cc:654] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302: Neither blocks nor log segments found. Creating new log.
I20260618 22:30:33.690267 27181 raft_consensus.cc:348] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } } peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } }
I20260618 22:30:33.690908 27181 raft_consensus.cc:374] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:30:33.691058 27181 raft_consensus.cc:729] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f3eb64d664474cbf8679f78a965f2b, State: Initialized, Role: FOLLOWER
I20260618 22:30:33.691475 27184 log.cc:824] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302: Log is configured to *not* fsync() on all Append() calls
I20260618 22:30:33.691692 27183 tablet_bootstrap.cc:492] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977: No bootstrap required, opened a new log
I20260618 22:30:33.691985 27183 ts_tablet_manager.cc:1397] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260618 22:30:33.691921 27181 consensus_queue.cc:260] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } } peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } }
I20260618 22:30:33.693815 27181 ts_tablet_manager.cc:1428] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260618 22:30:33.694269 27184 tablet_bootstrap.cc:492] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302: No bootstrap required, opened a new log
I20260618 22:30:33.694290 27163 heartbeater.cc:503] Master 127.10.242.254:40485 was elected leader, sending a full tablet report...
I20260618 22:30:33.694561 27184 ts_tablet_manager.cc:1397] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260618 22:30:33.700845 27183 raft_consensus.cc:348] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } } peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } }
I20260618 22:30:33.701361 27183 raft_consensus.cc:374] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:30:33.701473 27183 raft_consensus.cc:729] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d8b95883e6047f09c698c5f55f90977, State: Initialized, Role: FOLLOWER
I20260618 22:30:33.702224 27183 consensus_queue.cc:260] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } } peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } }
I20260618 22:30:33.702648 27184 raft_consensus.cc:348] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } } peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } }
I20260618 22:30:33.703152 27184 raft_consensus.cc:374] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:30:33.703260 27184 raft_consensus.cc:729] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc19bc4ce9844a68a5401b9ee888f302, State: Initialized, Role: FOLLOWER
I20260618 22:30:33.704026 27183 ts_tablet_manager.cc:1428] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977: Time spent starting tablet: real 0.012s user 0.001s sys 0.010s
I20260618 22:30:33.704010 27184 consensus_queue.cc:260] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } } peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } }
I20260618 22:30:33.705137 27030 heartbeater.cc:503] Master 127.10.242.254:40485 was elected leader, sending a full tablet report...
I20260618 22:30:33.705758 27184 ts_tablet_manager.cc:1428] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302: Time spent starting tablet: real 0.011s user 0.007s sys 0.005s
I20260618 22:30:33.738751 27188 raft_consensus.cc:482] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:30:33.739058 27188 raft_consensus.cc:504] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } } peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } }
I20260618 22:30:33.740849 27188 leader_election.cc:288] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0f3eb64d664474cbf8679f78a965f2b (127.10.242.195:43709), dc19bc4ce9844a68a5401b9ee888f302 (127.10.242.194:43839)
I20260618 22:30:33.748883 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:33.742258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:47169 (local address 127.10.242.194:43839)
0618 22:30:33.742454 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:33.742460 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:33.742689 (+ 229us) server_negotiation.cc:407] Connection header received
0618 22:30:33.742895 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:33.742899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:33.742958 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:33.743039 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:33.743527 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.744410 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.746676 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.747831 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.747962 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:33.748027 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:33.748475 (+ 448us) server_negotiation.cc:299] Negotiation successful
0618 22:30:33.748641 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:30:33.748883 27176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:33.741244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:42421 (local address 127.10.242.195:43709)
0618 22:30:33.741465 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:33.741471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:33.741961 (+ 490us) server_negotiation.cc:407] Connection header received
0618 22:30:33.742254 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:33.742261 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:33.742345 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:33.742412 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:33.743062 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.744029 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.746545 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.747728 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.747900 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:33.747977 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:33.748475 (+ 498us) server_negotiation.cc:299] Negotiation successful
0618 22:30:33.748622 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260618 22:30:33.749055 27190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:33.741194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:43709 (local address 127.10.242.193:42421)
0618 22:30:33.741862 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:33.741881 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:33.742017 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:33.742608 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:33.742611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:33.742633 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:33.742912 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:33.742920 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.744162 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.744165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:33.746364 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:33.746380 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.747937 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.747944 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:33.748201 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:33.748312 (+ 111us) client_negotiation.cc:769] Sending connection context
0618 22:30:33.748411 (+ 99us) client_negotiation.cc:240] Negotiation successful
0618 22:30:33.748602 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":90,"thread_start_us":330,"threads_started":1}
I20260618 22:30:33.749238 27191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:33.742009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:43839 (local address 127.10.242.193:47169)
0618 22:30:33.742594 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:33.742608 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:33.742752 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:33.743073 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:33.743076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:33.743095 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:33.743386 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:33.743394 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.744528 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.744533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:33.746545 (+ 2012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:33.746557 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.748010 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.748016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:33.748275 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:33.748325 (+ 50us) client_negotiation.cc:769] Sending connection context
0618 22:30:33.748411 (+ 86us) client_negotiation.cc:240] Negotiation successful
0618 22:30:33.748602 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":80,"thread_start_us":267,"threads_started":1}
I20260618 22:30:33.750279 26985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e002cb45759f4035a812ea083402a79c" candidate_uuid: "7d8b95883e6047f09c698c5f55f90977" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc19bc4ce9844a68a5401b9ee888f302" is_pre_election: true
I20260618 22:30:33.750243 27118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e002cb45759f4035a812ea083402a79c" candidate_uuid: "7d8b95883e6047f09c698c5f55f90977" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f3eb64d664474cbf8679f78a965f2b" is_pre_election: true
I20260618 22:30:33.750834 26985 raft_consensus.cc:2454] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d8b95883e6047f09c698c5f55f90977 in term 0.
I20260618 22:30:33.750834 27118 raft_consensus.cc:2454] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d8b95883e6047f09c698c5f55f90977 in term 0.
I20260618 22:30:33.751530 26785 leader_election.cc:302] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d8b95883e6047f09c698c5f55f90977, dc19bc4ce9844a68a5401b9ee888f302; no voters:
I20260618 22:30:33.751904 27188 raft_consensus.cc:2781] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:30:33.752056 27188 raft_consensus.cc:482] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:30:33.752163 27188 raft_consensus.cc:3037] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:30:33.754395 27188 raft_consensus.cc:504] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } } peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } }
I20260618 22:30:33.754989 27188 leader_election.cc:288] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [CANDIDATE]: Term 1 election: Requested vote from peers d0f3eb64d664474cbf8679f78a965f2b (127.10.242.195:43709), dc19bc4ce9844a68a5401b9ee888f302 (127.10.242.194:43839)
I20260618 22:30:33.755487 27118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e002cb45759f4035a812ea083402a79c" candidate_uuid: "7d8b95883e6047f09c698c5f55f90977" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f3eb64d664474cbf8679f78a965f2b"
I20260618 22:30:33.755568 26985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e002cb45759f4035a812ea083402a79c" candidate_uuid: "7d8b95883e6047f09c698c5f55f90977" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc19bc4ce9844a68a5401b9ee888f302"
I20260618 22:30:33.755674 27118 raft_consensus.cc:3037] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:30:33.755775 26985 raft_consensus.cc:3037] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:30:33.757922 27118 raft_consensus.cc:2454] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d8b95883e6047f09c698c5f55f90977 in term 1.
I20260618 22:30:33.758047 26985 raft_consensus.cc:2454] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d8b95883e6047f09c698c5f55f90977 in term 1.
I20260618 22:30:33.758332 26785 leader_election.cc:302] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d8b95883e6047f09c698c5f55f90977, d0f3eb64d664474cbf8679f78a965f2b; no voters:
I20260618 22:30:33.758679 27188 raft_consensus.cc:2781] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:30:33.759102 27188 raft_consensus.cc:686] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [term 1 LEADER]: Becoming Leader. State: Replica: 7d8b95883e6047f09c698c5f55f90977, State: Running, Role: LEADER
I20260618 22:30:33.759589 27188 consensus_queue.cc:237] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } } peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } }
I20260618 22:30:33.764611 26707 catalog_manager.cc:5495] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d8b95883e6047f09c698c5f55f90977 (127.10.242.193). New cstate: current_term: 1 leader_uuid: "7d8b95883e6047f09c698c5f55f90977" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } health_report { overall_health: HEALTHY } } }
I20260618 22:30:33.808859 27180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:33.803298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43526 (local address 127.10.242.193:39701)
0618 22:30:33.803527 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:33.803533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:33.803553 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:30:33.803823 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:33.803827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:33.803923 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:33.804126 (+ 203us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:30:33.804478 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.805262 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.806728 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.807179 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.807288 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:33.807771 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:30:33.808293 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:30:33.808426 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:33.808520 (+ 94us) server_negotiation.cc:299] Negotiation successful
0618 22:30:33.808605 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260618 22:30:33.810561 27164 tablet.cc:2369] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:30:33.811146 27031 tablet.cc:2369] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:30:33.817524 27118 raft_consensus.cc:1261] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b [term 1 FOLLOWER]: Refusing update from remote peer 7d8b95883e6047f09c698c5f55f90977: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:30:33.817525 26985 raft_consensus.cc:1261] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302 [term 1 FOLLOWER]: Refusing update from remote peer 7d8b95883e6047f09c698c5f55f90977: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:30:33.818414 27192 consensus_queue.cc:1035] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:30:33.818787 27188 consensus_queue.cc:1035] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 22:30:33.821256 26897 tablet.cc:2369] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:30:33.829790 27195 mvcc.cc:205] Tried to move back new op lower bound from 7298342231306797056 to 7298342231083151360. Current Snapshot: MvccSnapshot[applied={T|T < 7298342231306797056}]
I20260618 22:30:33.830838 27196 mvcc.cc:205] Tried to move back new op lower bound from 7298342231306797056 to 7298342231083151360. Current Snapshot: MvccSnapshot[applied={T|T < 7298342231306797056}]
I20260618 22:30:33.844208 26707 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:30:33.844463 26707 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:30:33.847878 26707 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: Sending DeleteTablet for 3 replicas of tablet e002cb45759f4035a812ea083402a79c
I20260618 22:30:33.849015 27098 tablet_service.cc:1514] Processing DeleteTablet for tablet e002cb45759f4035a812ea083402a79c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:30:33 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:52152
I20260618 22:30:33.849376 26965 tablet_service.cc:1514] Processing DeleteTablet for tablet e002cb45759f4035a812ea083402a79c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:30:33 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:36258
I20260618 22:30:33.849424 26831 tablet_service.cc:1514] Processing DeleteTablet for tablet e002cb45759f4035a812ea083402a79c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:30:33 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:43512
I20260618 22:30:33.850097 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 26768
I20260618 22:30:33.859378 27206 tablet_replica.cc:330] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302: stopping tablet replica
I20260618 22:30:33.860679 27206 raft_consensus.cc:2229] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:30:33.861994 27206 raft_consensus.cc:2258] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:30:33.862901 27205 tablet_replica.cc:330] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b: stopping tablet replica
I20260618 22:30:33.863404 27205 raft_consensus.cc:2229] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:30:33.863476 27207 tablet_replica.cc:330] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977: stopping tablet replica
I20260618 22:30:33.863906 27207 raft_consensus.cc:2229] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:30:33.864670 27207 raft_consensus.cc:2258] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:30:33.865432 27206 ts_tablet_manager.cc:1905] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:30:33.865614 27205 raft_consensus.cc:2258] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:30:33.866729 27207 ts_tablet_manager.cc:1905] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:30:33.868927 27205 ts_tablet_manager.cc:1905] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:30:33.872488 27206 ts_tablet_manager.cc:1918] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:30:33.872749 27206 log.cc:1196] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/e002cb45759f4035a812ea083402a79c
I20260618 22:30:33.873335 27206 ts_tablet_manager.cc:1939] T e002cb45759f4035a812ea083402a79c P dc19bc4ce9844a68a5401b9ee888f302: Deleting consensus metadata
I20260618 22:30:33.875564 27207 ts_tablet_manager.cc:1918] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:30:33.875551 26692 catalog_manager.cc:4841] TS dc19bc4ce9844a68a5401b9ee888f302 (127.10.242.194:43839): tablet e002cb45759f4035a812ea083402a79c (table test-table [id=ae6cbfbd1a9241a2b5c119d3858099c3]) successfully deleted
I20260618 22:30:33.875789 27207 log.cc:1196] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/e002cb45759f4035a812ea083402a79c
I20260618 22:30:33.876340 27207 ts_tablet_manager.cc:1939] T e002cb45759f4035a812ea083402a79c P 7d8b95883e6047f09c698c5f55f90977: Deleting consensus metadata
I20260618 22:30:33.878068 27205 ts_tablet_manager.cc:1918] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:30:33.878252 27205 log.cc:1196] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/e002cb45759f4035a812ea083402a79c
I20260618 22:30:33.878316 26692 catalog_manager.cc:4841] TS 7d8b95883e6047f09c698c5f55f90977 (127.10.242.193:39701): tablet e002cb45759f4035a812ea083402a79c (table test-table [id=ae6cbfbd1a9241a2b5c119d3858099c3]) successfully deleted
I20260618 22:30:33.878746 27205 ts_tablet_manager.cc:1939] T e002cb45759f4035a812ea083402a79c P d0f3eb64d664474cbf8679f78a965f2b: Deleting consensus metadata
I20260618 22:30:33.880672 26692 catalog_manager.cc:4841] TS d0f3eb64d664474cbf8679f78a965f2b (127.10.242.195:43709): tablet e002cb45759f4035a812ea083402a79c (table test-table [id=ae6cbfbd1a9241a2b5c119d3858099c3]) successfully deleted
Jun 18 22:30:33 dist-test-slave-0956 krb5kdc[26662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:30:33.889678 27180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:33.851306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43536 (local address 127.10.242.193:39701)
0618 22:30:33.853939 (+ 2633us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:33.853946 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:33.859747 (+ 5801us) server_negotiation.cc:407] Connection header received
0618 22:30:33.860085 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:33.860093 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:33.860186 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:33.860255 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:33.863705 (+ 3450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.864942 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.870189 (+ 5247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:33.870934 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:33.871058 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:33.882809 (+ 11751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:33.882824 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:33.882839 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:33.882888 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:33.886397 (+ 3509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:33.887378 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:33.887386 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:33.887388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:33.887556 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:33.887959 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:33.887963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:33.887969 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:33.888352 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:33.888651 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:33.889009 (+ 358us) server_negotiation.cc:299] Negotiation successful
0618 22:30:33.889339 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2498}
I20260618 22:30:33.890358 26891 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:33.991410 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 26768
I20260618 22:30:34.006819 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 26902
Jun 18 22:30:34 dist-test-slave-0956 krb5kdc[26662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.194@KRBTEST.COM
I20260618 22:30:34.022135 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:34.007598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36272 (local address 127.10.242.194:43839)
0618 22:30:34.007784 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:34.007790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:34.007810 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:30:34.007995 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:34.008000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:34.008065 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:34.008147 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:34.009497 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:34.010219 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:34.011687 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:34.012117 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:34.012232 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:34.016545 (+ 4313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:34.016561 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:34.016577 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:34.016628 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:34.019389 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:34.020051 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:34.020058 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:34.020061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:34.020191 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:34.020567 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:34.020571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:34.020573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:34.020948 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:34.021284 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:34.021558 (+ 274us) server_negotiation.cc:299] Negotiation successful
0618 22:30:34.021778 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:30:34.022800 27025 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:34.111038 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 26902
I20260618 22:30:34.124583 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 27035
Jun 18 22:30:34 dist-test-slave-0956 krb5kdc[26662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.195@KRBTEST.COM
I20260618 22:30:34.139780 27176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:34.125370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52158 (local address 127.10.242.195:43709)
0618 22:30:34.125571 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:34.125575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:34.125594 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:30:34.125725 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:34.125729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:34.125792 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:34.125872 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:34.127137 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:34.127928 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:34.129284 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:34.129708 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:34.129804 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:34.134174 (+ 4370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:34.134193 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:34.134208 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:34.134255 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:34.136952 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:34.137684 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:34.137692 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:34.137695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:34.137862 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:34.138228 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:34.138232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:34.138234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:34.138599 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:34.138816 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:34.139274 (+ 458us) server_negotiation.cc:299] Negotiation successful
0618 22:30:34.139455 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260618 22:30:34.140434 27158 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:34.229068 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 27035
I20260618 22:30:34.241954 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 26675
I20260618 22:30:34.243058 26737 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:34.351384 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 26675
I20260618 22:30:54.368774 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:40485
--webserver_interface=127.10.242.254
--webserver_port=35479
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:40485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:30:54.694123 27213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:54.694442 27213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:54.694551 27213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:54.702268 27213 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260618 22:30:54.702383 27213 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260618 22:30:54.702431 27213 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260618 22:30:54.702503 27213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:30:54.702560 27213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:54.702658 27213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:54.702713 27213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:30:54.702786 27213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:30:54.713161 27213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:40485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:40485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=35479
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:54.714483 27213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:54.715888 27213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:54.738359 27222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:54.738386 27219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:54.738416 27220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:54.739264 27213 server_base.cc:1034] running on GCE node
I20260618 22:30:54.740162 27213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:54.741904 27213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:54.743148 27213 hybrid_clock.cc:648] HybridClock initialized: now 1781821854743086 us; error 35 us; skew 500 ppm
I20260618 22:30:54.743546 27213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:54 dist-test-slave-0956 krb5kdc[26662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821854, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:54.747301 27213 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:30:54.749217 27213 webserver.cc:460] Webserver started at http://127.10.242.254:35479/ using document root <none> and password file <none>
I20260618 22:30:54.750029 27213 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:54.750172 27213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:54.757319 27213 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.002s
I20260618 22:30:54.760761 27229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:54.762391 27213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:30:54.762565 27213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "b8c6ff6457674018b0ccbf5a46d88e4b"
format_stamp: "Formatted at 2026-06-18 22:30:31 on dist-test-slave-0956"
server_key: "7b1f9a1ae2a5b094ee5efbe16fd18959"
server_key_iv: "084e16f71dcf488fb9c884ab19e471cb"
server_key_version: "encryptionkey@0"
I20260618 22:30:54.763622 27213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:54.781275 27213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:54.785696 27213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:54.785962 27213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:54.803599 27213 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:40485
I20260618 22:30:54.803618 27280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:40485 every 8 connection(s)
I20260618 22:30:54.805171 27213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:30:54.811491 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 27213
I20260618 22:30:54.812323 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:39701
--local_ip_for_outbound_sockets=127.10.242.193
--tserver_master_addrs=127.10.242.254:40485
--webserver_port=44709
--webserver_interface=127.10.242.193
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:30:54.812526 27281 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:30:54.816250 27281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: Bootstrap starting.
I20260618 22:30:54.841069 27281 log.cc:824] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:30:54.857693 27281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:30:54.858570 27281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: Bootstrap complete.
I20260618 22:30:54.873653 27281 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40485 } }
I20260618 22:30:54.874907 27281 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c6ff6457674018b0ccbf5a46d88e4b, State: Initialized, Role: FOLLOWER
I20260618 22:30:54.875643 27281 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40485 } }
I20260618 22:30:54.875958 27281 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:30:54.876081 27281 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:30:54.876256 27281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:30:54.881484 27281 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40485 } }
I20260618 22:30:54.882105 27281 leader_election.cc:302] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8c6ff6457674018b0ccbf5a46d88e4b; no voters:
I20260618 22:30:54.882704 27281 leader_election.cc:288] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260618 22:30:54.883411 27285 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 2 FOLLOWER]: Leader election won for term 2
I20260618 22:30:54.884030 27285 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 2 LEADER]: Becoming Leader. State: Replica: b8c6ff6457674018b0ccbf5a46d88e4b, State: Running, Role: LEADER
I20260618 22:30:54.884598 27285 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40485 } }
I20260618 22:30:54.884756 27281 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:30:54.887037 27287 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8c6ff6457674018b0ccbf5a46d88e4b. Latest consensus state: current_term: 2 leader_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40485 } } }
I20260618 22:30:54.887423 27287 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [sys.catalog]: This master's current role is: LEADER
I20260618 22:30:54.887938 27286 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40485 } } }
I20260618 22:30:54.888209 27286 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [sys.catalog]: This master's current role is: LEADER
I20260618 22:30:54.894948 27291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:30:54.904531 27291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:30:54.906297 27291 catalog_manager.cc:1259] Loaded cluster ID: 8e89f0d1bb6e47c68173ab68c89b3fe1
I20260618 22:30:54.906509 27291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:30:54.911281 27291 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:30:54.913215 27291 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: Loaded TSK: 0
I20260618 22:30:54.963485 27291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: Generated new TSK 1
I20260618 22:30:54.963932 27291 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260618 22:30:55.196979 27283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:55.197374 27283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:55.197535 27283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:55.205456 27283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:55.205598 27283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:55.205754 27283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:30:55.215844 27283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:39701
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=44709
--tserver_master_addrs=127.10.242.254:40485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:55.217211 27283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:55.218515 27283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:55.233716 27309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:55.235524 27310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:55.238500 27312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:55.238474 27283 server_base.cc:1034] running on GCE node
I20260618 22:30:55.239440 27283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:55.240625 27283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:55.241935 27283 hybrid_clock.cc:648] HybridClock initialized: now 1781821855241841 us; error 84 us; skew 500 ppm
I20260618 22:30:55.242309 27283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:55 dist-test-slave-0956 krb5kdc[26662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821855, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:55.247435 27283 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:30:55.249357 27283 webserver.cc:460] Webserver started at http://127.10.242.193:44709/ using document root <none> and password file <none>
I20260618 22:30:55.250315 27283 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:55.250525 27283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:55.257894 27283 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260618 22:30:55.261530 27319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:55.263449 27283 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260618 22:30:55.263652 27283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "7d8b95883e6047f09c698c5f55f90977"
format_stamp: "Formatted at 2026-06-18 22:30:32 on dist-test-slave-0956"
server_key: "32fb911939ca83d16524223987e420b1"
server_key_iv: "48c372ad792a680226d076523c6251e9"
server_key_version: "encryptionkey@0"
I20260618 22:30:55.265030 27283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:55.297353 27283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:55.301604 27283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:55.301863 27283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:55.302881 27283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:30:55.304195 27283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:30:55.304296 27283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:55.304400 27283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:30:55.304476 27283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:55.335530 27283 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:39701
I20260618 22:30:55.335599 27431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:39701 every 8 connection(s)
I20260618 22:30:55.336948 27283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:30:55.342342 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 27283
I20260618 22:30:55.342932 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:43839
--local_ip_for_outbound_sockets=127.10.242.194
--tserver_master_addrs=127.10.242.254:40485
--webserver_port=39637
--webserver_interface=127.10.242.194
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:30:55 dist-test-slave-0956 krb5kdc[26662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821855, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:30:55.360930 27435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:55.340097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:56913)
0618 22:30:55.340874 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:55.340912 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:55.342749 (+ 1837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:55.344181 (+ 1432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:55.344191 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:55.344784 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:55.345380 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:55.345399 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:55.346994 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:55.346998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:55.348257 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:55.348271 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:55.349685 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:55.349692 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:55.349908 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:55.350592 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:55.350616 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:55.352971 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:55.356463 (+ 3492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:55.356472 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:55.356488 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:55.356942 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:55.357433 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:55.357438 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:55.357441 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:55.357665 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:55.358406 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:55.358414 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:55.358768 (+ 354us) client_negotiation.cc:769] Sending connection context
0618 22:30:55.359016 (+ 248us) client_negotiation.cc:240] Negotiation successful
0618 22:30:55.359606 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":300,"threads_started":1}
I20260618 22:30:55.360930 27434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:55.339726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:56913 (local address 127.10.242.254:40485)
0618 22:30:55.340321 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:55.340334 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:55.341055 (+ 721us) server_negotiation.cc:407] Connection header received
0618 22:30:55.343315 (+ 2260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:55.343329 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:55.343684 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:55.344008 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:55.345564 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:55.346810 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:55.348406 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:55.349416 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:55.349609 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:55.353304 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:55.353313 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:55.353330 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:55.353375 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:55.356302 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:55.357124 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:55.357134 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:55.357141 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:55.357318 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:55.357775 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:55.357779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:55.357781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:55.358272 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:55.358452 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:55.359256 (+ 804us) server_negotiation.cc:299] Negotiation successful
0618 22:30:55.359656 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":252,"threads_started":1}
I20260618 22:30:55.363981 27432 heartbeater.cc:347] Connected to a master server at 127.10.242.254:40485
I20260618 22:30:55.364327 27432 heartbeater.cc:464] Registering TS with master...
I20260618 22:30:55.365221 27432 heartbeater.cc:511] Master 127.10.242.254:40485 requested a full tablet report, sending...
I20260618 22:30:55.367671 27246 ts_manager.cc:194] Registered new tserver with Master: 7d8b95883e6047f09c698c5f55f90977 (127.10.242.193:39701)
I20260618 22:30:55.369338 27246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:56913
W20260618 22:30:55.667189 27436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:55.667541 27436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:55.667671 27436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:55.674965 27436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:55.675076 27436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:55.675222 27436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:30:55.685098 27436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:43839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=39637
--tserver_master_addrs=127.10.242.254:40485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:55.686511 27436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:55.687844 27436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:55.701284 27442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:55.701254 27444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:55.701231 27441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:55.702104 27436 server_base.cc:1034] running on GCE node
I20260618 22:30:55.702708 27436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:55.703658 27436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:55.704870 27436 hybrid_clock.cc:648] HybridClock initialized: now 1781821855704809 us; error 46 us; skew 500 ppm
I20260618 22:30:55.705242 27436 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:55 dist-test-slave-0956 krb5kdc[26662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821855, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:55.708259 27436 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:30:55.709836 27436 webserver.cc:460] Webserver started at http://127.10.242.194:39637/ using document root <none> and password file <none>
I20260618 22:30:55.710551 27436 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:55.710681 27436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:55.716272 27436 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260618 22:30:55.719218 27451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:55.720793 27436 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:30:55.721029 27436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "dc19bc4ce9844a68a5401b9ee888f302"
format_stamp: "Formatted at 2026-06-18 22:30:32 on dist-test-slave-0956"
server_key: "5501cc29c30685023820244dd6eb1af2"
server_key_iv: "34ca8ff20a138913d0d6084b8faa89d7"
server_key_version: "encryptionkey@0"
I20260618 22:30:55.722131 27436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:55.739662 27436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:55.744437 27436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:55.744731 27436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:55.746004 27436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:30:55.747440 27436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:30:55.747550 27436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:55.747674 27436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:30:55.747746 27436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:55.780215 27436 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:43839
I20260618 22:30:55.780306 27563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:43839 every 8 connection(s)
I20260618 22:30:55.781771 27436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:30:55.786294 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 27436
I20260618 22:30:55.786914 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:43709
--local_ip_for_outbound_sockets=127.10.242.195
--tserver_master_addrs=127.10.242.254:40485
--webserver_port=41263
--webserver_interface=127.10.242.195
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:30:55 dist-test-slave-0956 krb5kdc[26662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821855, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:30:55.803892 27434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:55.784359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:40835 (local address 127.10.242.254:40485)
0618 22:30:55.784536 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:55.784541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:55.785795 (+ 1254us) server_negotiation.cc:407] Connection header received
0618 22:30:55.787799 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:55.787805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:55.787898 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:55.788009 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:55.789744 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:55.790743 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:55.792413 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:55.793003 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:55.793153 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:55.797277 (+ 4124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:55.797283 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:55.797287 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:55.797326 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:55.800221 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:55.801271 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:55.801277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:55.801281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:55.801426 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:55.801923 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:55.801926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:55.801928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:55.802206 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:55.802325 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:55.803281 (+ 956us) server_negotiation.cc:299] Negotiation successful
0618 22:30:55.803552 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:30:55.806221 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:55.784853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:40835)
0618 22:30:55.785625 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:55.785665 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:55.787499 (+ 1834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:55.788329 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:55.788339 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:55.788886 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:55.789538 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:55.789557 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:55.790924 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:55.790928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:55.792266 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:55.792280 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:55.793278 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:55.793285 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:55.793510 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:55.794191 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:55.794215 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:55.796967 (+ 2752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:55.800429 (+ 3462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:55.800440 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:55.800460 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:55.801116 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:55.801544 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:55.801549 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:55.801553 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:55.801826 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:55.802318 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:55.802326 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:55.802830 (+ 504us) client_negotiation.cc:769] Sending connection context
0618 22:30:55.804139 (+ 1309us) client_negotiation.cc:240] Negotiation successful
0618 22:30:55.804747 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":292,"threads_started":1}
I20260618 22:30:55.807961 27564 heartbeater.cc:347] Connected to a master server at 127.10.242.254:40485
I20260618 22:30:55.808301 27564 heartbeater.cc:464] Registering TS with master...
I20260618 22:30:55.809239 27564 heartbeater.cc:511] Master 127.10.242.254:40485 requested a full tablet report, sending...
I20260618 22:30:55.811026 27246 ts_manager.cc:194] Registered new tserver with Master: dc19bc4ce9844a68a5401b9ee888f302 (127.10.242.194:43839)
I20260618 22:30:55.812233 27246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:40835
W20260618 22:30:56.115052 27567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:30:56.115339 27567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:30:56.115495 27567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:30:56.122776 27567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:30:56.122895 27567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:30:56.123041 27567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:30:56.132848 27567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:43709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=41263
--tserver_master_addrs=127.10.242.254:40485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:30:56.134155 27567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:30:56.135490 27567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:30:56.148833 27572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:56.148838 27573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:30:56.149047 27575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:30:56.149555 27567 server_base.cc:1034] running on GCE node
I20260618 22:30:56.150170 27567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:30:56.151120 27567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:30:56.152328 27567 hybrid_clock.cc:648] HybridClock initialized: now 1781821856152268 us; error 54 us; skew 500 ppm
I20260618 22:30:56.152640 27567 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:30:56 dist-test-slave-0956 krb5kdc[26662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821856, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:30:56.160320 27567 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:30:56.161962 27567 webserver.cc:460] Webserver started at http://127.10.242.195:41263/ using document root <none> and password file <none>
I20260618 22:30:56.162654 27567 fs_manager.cc:359] Metadata directory not provided
I20260618 22:30:56.162794 27567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:30:56.168843 27567 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260618 22:30:56.171948 27582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:56.173403 27567 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:30:56.173559 27567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "d0f3eb64d664474cbf8679f78a965f2b"
format_stamp: "Formatted at 2026-06-18 22:30:33 on dist-test-slave-0956"
server_key: "90dd29a27cdf9e8e36d4506069f9590a"
server_key_iv: "b4aef2d90f23d9b1e91055358e7a127c"
server_key_version: "encryptionkey@0"
I20260618 22:30:56.174520 27567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:30:56.200806 27567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:30:56.205287 27567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:30:56.205551 27567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:30:56.206596 27567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:30:56.207962 27567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:30:56.208056 27567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:56.208163 27567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:30:56.208231 27567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:30:56.238555 27567 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:43709
I20260618 22:30:56.238595 27694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:43709 every 8 connection(s)
I20260618 22:30:56.239992 27567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:30:56.242780 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 27567
Jun 18 22:30:56 dist-test-slave-0956 krb5kdc[26662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821856, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:30:56.260772 27434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.242664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:37723 (local address 127.10.242.254:40485)
0618 22:30:56.242896 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:56.242902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:56.244112 (+ 1210us) server_negotiation.cc:407] Connection header received
0618 22:30:56.246225 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:56.246230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:56.246311 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:56.246397 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:56.247912 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.248621 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.250731 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.251162 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.251261 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.255067 (+ 3806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:56.255072 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:56.255075 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:56.255106 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:56.257784 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:56.258511 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:56.258515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:56.258517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:56.258636 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:56.259055 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:56.259059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:56.259061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:56.259353 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:56.259498 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:56.260182 (+ 684us) server_negotiation.cc:299] Negotiation successful
0618 22:30:56.260402 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260618 22:30:56.262040 27697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.243156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:37723)
0618 22:30:56.243955 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:56.244000 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:56.245950 (+ 1950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:56.246645 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:56.246654 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:56.247162 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:30:56.247744 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:56.247762 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.248781 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.248785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:56.250571 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:56.250586 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.251385 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.251394 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:56.251681 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.252357 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:30:56.252380 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:30:56.254856 (+ 2476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:30:56.257941 (+ 3085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:56.257950 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:56.257967 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:56.258389 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:56.258752 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:30:56.258757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:30:56.258761 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:30:56.258943 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:30:56.259481 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:30:56.259489 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:30:56.259829 (+ 340us) client_negotiation.cc:769] Sending connection context
0618 22:30:56.260067 (+ 238us) client_negotiation.cc:240] Negotiation successful
0618 22:30:56.260617 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":320,"threads_started":1}
I20260618 22:30:56.262856 27699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.243684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46600 (local address 127.10.242.254:40485)
0618 22:30:56.245454 (+ 1770us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:56.245460 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:56.249636 (+ 4176us) server_negotiation.cc:407] Connection header received
0618 22:30:56.249820 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:56.249823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:56.249880 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:56.249966 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:56.251897 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.252600 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.254243 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.254669 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.254780 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.257703 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:56.257708 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:56.257711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:56.257777 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:56.260126 (+ 2349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:56.260806 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:56.260810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:56.260812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:56.260972 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:56.261408 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:56.261412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:56.261414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:56.261651 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:56.261831 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:56.262260 (+ 429us) server_negotiation.cc:299] Negotiation successful
0618 22:30:56.262473 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1657,"spinlock_wait_cycles":30592,"thread_start_us":1544,"threads_started":1}
I20260618 22:30:56.263695 27695 heartbeater.cc:347] Connected to a master server at 127.10.242.254:40485
I20260618 22:30:56.264020 27695 heartbeater.cc:464] Registering TS with master...
I20260618 22:30:56.264725 27695 heartbeater.cc:511] Master 127.10.242.254:40485 requested a full tablet report, sending...
I20260618 22:30:56.266419 27246 ts_manager.cc:194] Registered new tserver with Master: d0f3eb64d664474cbf8679f78a965f2b (127.10.242.195:43709)
I20260618 22:30:56.267302 27246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:37723
I20260618 22:30:56.278138 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:30:56.285238 27699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.279142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46602 (local address 127.10.242.254:40485)
0618 22:30:56.279341 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:56.279345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:56.279844 (+ 499us) server_negotiation.cc:407] Connection header received
0618 22:30:56.280014 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:56.280018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:56.280181 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:56.280312 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:30:56.280855 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.281635 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.283298 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.283800 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.283903 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.284430 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:30:56.284627 (+ 197us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0618 22:30:56.284960 (+ 333us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46602: token expired
Metrics: {"server-negotiator.queue_time_us":55}
W20260618 22:30:56.285362 27699 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46602: token expired
W20260618 22:30:56.285898 27166 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.242.254:40485)
I20260618 22:30:56.298507 27699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.286372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46614 (local address 127.10.242.254:40485)
0618 22:30:56.286549 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:56.286554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:56.286725 (+ 171us) server_negotiation.cc:407] Connection header received
0618 22:30:56.286857 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:56.286860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:56.286920 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:56.286994 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:56.288392 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.289221 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.290540 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.291098 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.291205 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.293803 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:56.293808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:56.293811 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:56.293847 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:56.296150 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:56.296746 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:56.296751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:56.296753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:56.296882 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:56.297269 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:56.297273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:56.297275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:56.297596 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:56.297775 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:56.298010 (+ 235us) server_negotiation.cc:299] Negotiation successful
0618 22:30:56.298204 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260618 22:30:56.302044 27246 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:30:56.309279 27246 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:30:56.331908 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.320854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55230 (local address 127.10.242.194:43839)
0618 22:30:56.321427 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:56.321437 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:56.321933 (+ 496us) server_negotiation.cc:407] Connection header received
0618 22:30:56.322031 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:56.322036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:56.322309 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:56.322499 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:56.324842 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.326414 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.329578 (+ 3164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.330798 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.330950 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.331128 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:56.331446 (+ 318us) server_negotiation.cc:299] Negotiation successful
0618 22:30:56.331612 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":315,"threads_started":1}
I20260618 22:30:56.331980 27706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.320899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:43839 (local address 127.0.0.1:55230)
0618 22:30:56.321611 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:56.321662 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:56.321832 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:56.322805 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:56.322814 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:56.322849 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:56.324449 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:56.324465 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.326783 (+ 2318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.326788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:56.329420 (+ 2632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:56.329436 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.331014 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.331021 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:56.331239 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.331276 (+ 37us) client_negotiation.cc:769] Sending connection context
0618 22:30:56.331389 (+ 113us) client_negotiation.cc:240] Negotiation successful
0618 22:30:56.331599 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":5,"thread_start_us":495,"threads_started":1}
I20260618 22:30:56.332257 27708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.322473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36430 (local address 127.10.242.195:43709)
0618 22:30:56.323425 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:56.323431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:56.323456 (+ 25us) server_negotiation.cc:407] Connection header received
0618 22:30:56.323523 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:56.323528 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:56.323690 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:56.323854 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:56.324721 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.326965 (+ 2244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.329505 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.330865 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.331586 (+ 721us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.331737 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:56.331798 (+ 61us) server_negotiation.cc:299] Negotiation successful
0618 22:30:56.331948 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806,"thread_start_us":673,"threads_started":1}
I20260618 22:30:56.332619 27707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.322129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:43709 (local address 127.0.0.1:36430)
0618 22:30:56.322663 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:56.322679 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:56.322776 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:56.323859 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:56.323863 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:56.323886 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:56.324537 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:56.324546 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.327121 (+ 2575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.327126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:56.329294 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:56.329308 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.331107 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.331118 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:56.331310 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.331344 (+ 34us) client_negotiation.cc:769] Sending connection context
0618 22:30:56.331406 (+ 62us) client_negotiation.cc:240] Negotiation successful
0618 22:30:56.332236 (+ 830us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":651,"thread_start_us":283,"threads_started":1}
I20260618 22:30:56.336500 27630 tablet_service.cc:1467] Processing CreateTablet for tablet 6f25bacc02be46f3af107e52b3dae806 (DEFAULT_TABLE table=test-table [id=2276a3e0a7df4374a59a889e919fa0a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:30:56.336526 27499 tablet_service.cc:1467] Processing CreateTablet for tablet 6f25bacc02be46f3af107e52b3dae806 (DEFAULT_TABLE table=test-table [id=2276a3e0a7df4374a59a889e919fa0a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:30:56.337898 27630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f25bacc02be46f3af107e52b3dae806. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:30:56.337998 27499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f25bacc02be46f3af107e52b3dae806. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:30:56.340356 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.322728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39701 (local address 127.0.0.1:55632)
0618 22:30:56.328324 (+ 5596us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:56.328343 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:56.328456 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:56.329687 (+ 1231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:56.329692 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:56.329719 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:56.330099 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:56.330111 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.332221 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.332226 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:56.333947 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:56.333962 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.338590 (+ 4628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.338597 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:56.338774 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.338813 (+ 39us) client_negotiation.cc:769] Sending connection context
0618 22:30:56.338890 (+ 77us) client_negotiation.cc:240] Negotiation successful
0618 22:30:56.339034 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5466,"thread_start_us":5331,"threads_started":1}
I20260618 22:30:56.341609 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.323171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55632 (local address 127.10.242.193:39701)
0618 22:30:56.328914 (+ 5743us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:56.328923 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:56.328950 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:30:56.329040 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:56.329047 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:56.329328 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:56.330456 (+ 1128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:56.330664 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.332079 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.334324 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.338340 (+ 4016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.340775 (+ 2435us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.340951 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:56.341020 (+ 69us) server_negotiation.cc:299] Negotiation successful
0618 22:30:56.341228 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5582,"thread_start_us":5477,"threads_started":1}
I20260618 22:30:56.344722 27367 tablet_service.cc:1467] Processing CreateTablet for tablet 6f25bacc02be46f3af107e52b3dae806 (DEFAULT_TABLE table=test-table [id=2276a3e0a7df4374a59a889e919fa0a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:30:56.346073 27367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f25bacc02be46f3af107e52b3dae806. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:30:56.353899 27711 tablet_bootstrap.cc:492] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302: Bootstrap starting.
I20260618 22:30:56.356273 27712 tablet_bootstrap.cc:492] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b: Bootstrap starting.
I20260618 22:30:56.357565 27711 tablet_bootstrap.cc:654] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302: Neither blocks nor log segments found. Creating new log.
I20260618 22:30:56.358913 27711 log.cc:824] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302: Log is configured to *not* fsync() on all Append() calls
I20260618 22:30:56.359005 27713 tablet_bootstrap.cc:492] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977: Bootstrap starting.
I20260618 22:30:56.359462 27712 tablet_bootstrap.cc:654] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b: Neither blocks nor log segments found. Creating new log.
I20260618 22:30:56.360736 27712 log.cc:824] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:30:56.362591 27711 tablet_bootstrap.cc:492] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302: No bootstrap required, opened a new log
I20260618 22:30:56.362876 27711 ts_tablet_manager.cc:1397] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260618 22:30:56.362906 27713 tablet_bootstrap.cc:654] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977: Neither blocks nor log segments found. Creating new log.
I20260618 22:30:56.363785 27713 log.cc:824] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977: Log is configured to *not* fsync() on all Append() calls
I20260618 22:30:56.364056 27712 tablet_bootstrap.cc:492] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b: No bootstrap required, opened a new log
I20260618 22:30:56.364359 27712 ts_tablet_manager.cc:1397] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260618 22:30:56.367129 27713 tablet_bootstrap.cc:492] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977: No bootstrap required, opened a new log
I20260618 22:30:56.367348 27713 ts_tablet_manager.cc:1397] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260618 22:30:56.373297 27432 heartbeater.cc:503] Master 127.10.242.254:40485 was elected leader, sending a full tablet report...
I20260618 22:30:56.373106 27711 raft_consensus.cc:348] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } } peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } }
I20260618 22:30:56.373565 27711 raft_consensus.cc:374] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:30:56.373664 27711 raft_consensus.cc:729] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc19bc4ce9844a68a5401b9ee888f302, State: Initialized, Role: FOLLOWER
I20260618 22:30:56.374392 27711 consensus_queue.cc:260] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } } peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } }
I20260618 22:30:56.374531 27712 raft_consensus.cc:348] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } } peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } }
I20260618 22:30:56.374966 27712 raft_consensus.cc:374] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:30:56.375119 27712 raft_consensus.cc:729] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f3eb64d664474cbf8679f78a965f2b, State: Initialized, Role: FOLLOWER
I20260618 22:30:56.375615 27713 raft_consensus.cc:348] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } } peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } }
I20260618 22:30:56.376119 27713 raft_consensus.cc:374] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:30:56.375921 27712 consensus_queue.cc:260] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } } peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } }
I20260618 22:30:56.376271 27713 raft_consensus.cc:729] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d8b95883e6047f09c698c5f55f90977, State: Initialized, Role: FOLLOWER
I20260618 22:30:56.376926 27713 consensus_queue.cc:260] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } } peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } }
I20260618 22:30:56.382779 27711 ts_tablet_manager.cc:1428] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302: Time spent starting tablet: real 0.020s user 0.010s sys 0.009s
I20260618 22:30:56.383104 27695 heartbeater.cc:503] Master 127.10.242.254:40485 was elected leader, sending a full tablet report...
I20260618 22:30:56.383209 27564 heartbeater.cc:503] Master 127.10.242.254:40485 was elected leader, sending a full tablet report...
I20260618 22:30:56.384156 27712 ts_tablet_manager.cc:1428] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b: Time spent starting tablet: real 0.020s user 0.013s sys 0.007s
I20260618 22:30:56.385624 27713 ts_tablet_manager.cc:1428] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977: Time spent starting tablet: real 0.018s user 0.009s sys 0.009s
W20260618 22:30:56.491573 27696 tablet.cc:2369] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:30:56.521917 27717 raft_consensus.cc:482] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:30:56.522236 27717 raft_consensus.cc:504] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } } peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } }
I20260618 22:30:56.523839 27717 leader_election.cc:288] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0f3eb64d664474cbf8679f78a965f2b (127.10.242.195:43709), 7d8b95883e6047f09c698c5f55f90977 (127.10.242.193:39701)
I20260618 22:30:56.532244 27708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.524312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:39099 (local address 127.10.242.195:43709)
0618 22:30:56.524543 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:56.524547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:56.525063 (+ 516us) server_negotiation.cc:407] Connection header received
0618 22:30:56.525253 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:56.525259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:56.525345 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:56.525450 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:56.526250 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.527251 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.529748 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.530981 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.531168 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.531246 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:56.531827 (+ 581us) server_negotiation.cc:299] Negotiation successful
0618 22:30:56.531980 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260618 22:30:56.532244 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.525467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:55979 (local address 127.10.242.193:39701)
0618 22:30:56.525683 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:56.525705 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:56.525801 (+ 96us) server_negotiation.cc:407] Connection header received
0618 22:30:56.525988 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:56.525991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:56.526083 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:56.526159 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:30:56.526642 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.527568 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.529871 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.531059 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.531229 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.531300 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:56.531827 (+ 527us) server_negotiation.cc:299] Negotiation successful
0618 22:30:56.531980 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260618 22:30:56.532418 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.525120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39701 (local address 127.10.242.194:55979)
0618 22:30:56.525700 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:56.525716 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:56.525864 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:56.526178 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:56.526181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:56.526207 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:56.526491 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:56.526498 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.527687 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.527692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:56.529709 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:56.529720 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.531321 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.531326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:56.531581 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.531649 (+ 68us) client_negotiation.cc:769] Sending connection context
0618 22:30:56.531764 (+ 115us) client_negotiation.cc:240] Negotiation successful
0618 22:30:56.531934 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":60,"thread_start_us":283,"threads_started":1}
I20260618 22:30:56.532415 27720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.524254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:43709 (local address 127.10.242.194:39099)
0618 22:30:56.524931 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:56.524950 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:30:56.525115 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:30:56.525636 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:30:56.525641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:30:56.525668 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:30:56.526086 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:56.526100 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.527366 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.527369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:56.529529 (+ 2160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:30:56.529543 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.531318 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.531325 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:30:56.531599 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.531649 (+ 50us) client_negotiation.cc:769] Sending connection context
0618 22:30:56.531764 (+ 115us) client_negotiation.cc:240] Negotiation successful
0618 22:30:56.531926 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":58,"thread_start_us":371,"threads_started":1}
W20260618 22:30:56.533717 27565 tablet.cc:2369] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:30:56.533701 27387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f25bacc02be46f3af107e52b3dae806" candidate_uuid: "dc19bc4ce9844a68a5401b9ee888f302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d8b95883e6047f09c698c5f55f90977" is_pre_election: true
I20260618 22:30:56.533706 27650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f25bacc02be46f3af107e52b3dae806" candidate_uuid: "dc19bc4ce9844a68a5401b9ee888f302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f3eb64d664474cbf8679f78a965f2b" is_pre_election: true
I20260618 22:30:56.534277 27650 raft_consensus.cc:2454] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc19bc4ce9844a68a5401b9ee888f302 in term 0.
I20260618 22:30:56.534277 27387 raft_consensus.cc:2454] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc19bc4ce9844a68a5401b9ee888f302 in term 0.
I20260618 22:30:56.534946 27453 leader_election.cc:302] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0f3eb64d664474cbf8679f78a965f2b, dc19bc4ce9844a68a5401b9ee888f302; no voters:
I20260618 22:30:56.535318 27717 raft_consensus.cc:2781] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:30:56.535495 27717 raft_consensus.cc:482] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:30:56.535597 27717 raft_consensus.cc:3037] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:30:56.537796 27717 raft_consensus.cc:504] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } } peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } }
I20260618 22:30:56.538375 27717 leader_election.cc:288] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [CANDIDATE]: Term 1 election: Requested vote from peers d0f3eb64d664474cbf8679f78a965f2b (127.10.242.195:43709), 7d8b95883e6047f09c698c5f55f90977 (127.10.242.193:39701)
I20260618 22:30:56.538874 27650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f25bacc02be46f3af107e52b3dae806" candidate_uuid: "dc19bc4ce9844a68a5401b9ee888f302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f3eb64d664474cbf8679f78a965f2b"
I20260618 22:30:56.538951 27387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f25bacc02be46f3af107e52b3dae806" candidate_uuid: "dc19bc4ce9844a68a5401b9ee888f302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d8b95883e6047f09c698c5f55f90977"
I20260618 22:30:56.539100 27650 raft_consensus.cc:3037] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:30:56.539153 27387 raft_consensus.cc:3037] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:30:56.542275 27387 raft_consensus.cc:2454] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc19bc4ce9844a68a5401b9ee888f302 in term 1.
I20260618 22:30:56.542272 27650 raft_consensus.cc:2454] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc19bc4ce9844a68a5401b9ee888f302 in term 1.
I20260618 22:30:56.542794 27453 leader_election.cc:302] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0f3eb64d664474cbf8679f78a965f2b, dc19bc4ce9844a68a5401b9ee888f302; no voters:
I20260618 22:30:56.543076 27717 raft_consensus.cc:2781] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:30:56.543787 27717 raft_consensus.cc:686] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [term 1 LEADER]: Becoming Leader. State: Replica: dc19bc4ce9844a68a5401b9ee888f302, State: Running, Role: LEADER
I20260618 22:30:56.544245 27717 consensus_queue.cc:237] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } } peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } }
I20260618 22:30:56.549062 27243 catalog_manager.cc:5495] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc19bc4ce9844a68a5401b9ee888f302 (127.10.242.194). New cstate: current_term: 1 leader_uuid: "dc19bc4ce9844a68a5401b9ee888f302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc19bc4ce9844a68a5401b9ee888f302" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 43839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 } health_report { overall_health: UNKNOWN } } }
I20260618 22:30:56.575302 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.569829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55234 (local address 127.10.242.194:43839)
0618 22:30:56.570005 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:56.570009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:56.570027 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:30:56.570116 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:56.570119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:56.570204 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:56.570326 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:30:56.570777 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.571511 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.572909 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.573390 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.573500 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.574153 (+ 653us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:30:56.574719 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:30:56.574838 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:56.574948 (+ 110us) server_negotiation.cc:299] Negotiation successful
0618 22:30:56.575020 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:30:56.583788 27387 raft_consensus.cc:1261] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977 [term 1 FOLLOWER]: Refusing update from remote peer dc19bc4ce9844a68a5401b9ee888f302: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:30:56.583765 27650 raft_consensus.cc:1261] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b [term 1 FOLLOWER]: Refusing update from remote peer dc19bc4ce9844a68a5401b9ee888f302: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:30:56.584832 27722 consensus_queue.cc:1035] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d8b95883e6047f09c698c5f55f90977" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 39701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:30:56.585229 27717 consensus_queue.cc:1035] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f3eb64d664474cbf8679f78a965f2b" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 22:30:56.589236 27433 tablet.cc:2369] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:30:56.614363 27243 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:30:56.614715 27243 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:30:56.619719 27243 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: Sending DeleteTablet for 3 replicas of tablet 6f25bacc02be46f3af107e52b3dae806
I20260618 22:30:56.621542 27367 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f25bacc02be46f3af107e52b3dae806 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:30:56 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:55632
I20260618 22:30:56.621572 27630 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f25bacc02be46f3af107e52b3dae806 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:30:56 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:36430
I20260618 22:30:56.622386 27735 tablet_replica.cc:330] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b: stopping tablet replica
I20260618 22:30:56.622720 27499 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f25bacc02be46f3af107e52b3dae806 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:30:56 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:55230
I20260618 22:30:56.622882 27735 raft_consensus.cc:2229] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:30:56.623353 27735 raft_consensus.cc:2258] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:30:56.623986 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 27283
I20260618 22:30:56.625437 27735 ts_tablet_manager.cc:1905] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:30:56.633308 27735 ts_tablet_manager.cc:1918] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:30:56.633601 27735 log.cc:1196] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/6f25bacc02be46f3af107e52b3dae806
I20260618 22:30:56.633955 27736 tablet_replica.cc:330] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977: stopping tablet replica
I20260618 22:30:56.634308 27735 ts_tablet_manager.cc:1939] T 6f25bacc02be46f3af107e52b3dae806 P d0f3eb64d664474cbf8679f78a965f2b: Deleting consensus metadata
I20260618 22:30:56.634423 27737 tablet_replica.cc:330] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302: stopping tablet replica
I20260618 22:30:56.634617 27736 raft_consensus.cc:2229] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:30:56.634908 27737 raft_consensus.cc:2229] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:30:56.635171 27736 raft_consensus.cc:2258] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:30:56.635591 27737 raft_consensus.cc:2258] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:30:56.636462 27231 catalog_manager.cc:4841] TS d0f3eb64d664474cbf8679f78a965f2b (127.10.242.195:43709): tablet 6f25bacc02be46f3af107e52b3dae806 (table test-table [id=2276a3e0a7df4374a59a889e919fa0a0]) successfully deleted
I20260618 22:30:56.637305 27736 ts_tablet_manager.cc:1905] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:30:56.638170 27737 ts_tablet_manager.cc:1905] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:30:56.644351 27737 ts_tablet_manager.cc:1918] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:30:56.644364 27736 ts_tablet_manager.cc:1918] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:30:56.644445 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.624857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55638 (local address 127.10.242.193:39701)
0618 22:30:56.625232 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:56.625238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:56.625545 (+ 307us) server_negotiation.cc:407] Connection header received
0618 22:30:56.625614 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:56.625620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:56.625720 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:56.625807 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:56.627819 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.629111 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.631637 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.632365 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.632485 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.636170 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:56.636185 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:56.636200 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:56.636240 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:56.640087 (+ 3847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:56.641054 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:56.641064 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:56.641067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:56.641284 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:56.641825 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:56.641830 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:56.641833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:56.642339 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:56.642666 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:56.643826 (+ 1160us) server_negotiation.cc:299] Negotiation successful
0618 22:30:56.644072 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250}
I20260618 22:30:56.644537 27736 log.cc:1196] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/6f25bacc02be46f3af107e52b3dae806
I20260618 22:30:56.644793 27737 log.cc:1196] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/6f25bacc02be46f3af107e52b3dae806
I20260618 22:30:56.645128 27427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:56.645282 27737 ts_tablet_manager.cc:1939] T 6f25bacc02be46f3af107e52b3dae806 P dc19bc4ce9844a68a5401b9ee888f302: Deleting consensus metadata
I20260618 22:30:56.646811 27231 catalog_manager.cc:4841] TS dc19bc4ce9844a68a5401b9ee888f302 (127.10.242.194:43839): tablet 6f25bacc02be46f3af107e52b3dae806 (table test-table [id=2276a3e0a7df4374a59a889e919fa0a0]) successfully deleted
I20260618 22:30:56.737739 27736 ts_tablet_manager.cc:1939] T 6f25bacc02be46f3af107e52b3dae806 P 7d8b95883e6047f09c698c5f55f90977: Deleting consensus metadata
I20260618 22:30:56.738479 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 27283
W20260618 22:30:56.754277 27231 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.242.193:39701 (error 108)
I20260618 22:30:56.754509 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 27436
I20260618 22:30:56.756654 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.756106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39701 (local address 127.0.0.1:55640)
0618 22:30:56.756352 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:56.756478 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
W20260618 22:30:56.756927 27231 catalog_manager.cc:4584] TS 7d8b95883e6047f09c698c5f55f90977 (127.10.242.193:39701): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f25bacc02be46f3af107e52b3dae806: Network error: Client connection negotiation failed: client connection to 127.10.242.193:39701: connect: Connection refused (error 111)
I20260618 22:30:56.771487 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.755410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55248 (local address 127.10.242.194:43839)
0618 22:30:56.755593 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:56.755599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:56.755626 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:30:56.755803 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:56.755810 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:56.755893 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:56.756012 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:56.757703 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.758898 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.760176 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.760840 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.760977 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.764412 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:56.764429 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:56.764453 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:56.764512 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:56.768524 (+ 4012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:56.769308 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:56.769319 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:56.769322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:56.769509 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:56.769894 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:56.769904 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:56.769907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:56.770356 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:56.770697 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:56.770854 (+ 157us) server_negotiation.cc:299] Negotiation successful
0618 22:30:56.771080 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:30:56.772238 27559 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:56.810679 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.810179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39701 (local address 127.0.0.1:55656)
0618 22:30:56.810399 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:56.810516 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260618 22:30:56.811954 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.811458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39701 (local address 127.0.0.1:55670)
0618 22:30:56.811695 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:56.811796 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260618 22:30:56.872644 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.872181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39701 (local address 127.0.0.1:55672)
0618 22:30:56.872401 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:56.872492 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260618 22:30:56.873593 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.873268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39701 (local address 127.0.0.1:55688)
0618 22:30:56.873418 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:56.873488 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260618 22:30:56.883827 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 27436
I20260618 22:30:56.898703 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 27567
I20260618 22:30:56.915987 27708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.903278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36438 (local address 127.10.242.195:43709)
0618 22:30:56.903470 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0618 22:30:56.903475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:30:56.903496 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:30:56.903601 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:30:56.903605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:30:56.903681 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:30:56.903764 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:30:56.905181 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.905944 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.907259 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:30:56.907654 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:30:56.907750 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:30:56.910419 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:30:56.910435 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:30:56.910452 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:30:56.910488 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:30:56.913271 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:56.913996 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:56.914005 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:56.914007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:56.914152 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:30:56.914527 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:30:56.914532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:30:56.914534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:30:56.914915 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:30:56.915138 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0618 22:30:56.915461 (+ 323us) server_negotiation.cc:299] Negotiation successful
0618 22:30:56.915636 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260618 22:30:56.916638 27690 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:56.953928 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.953343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39701 (local address 127.0.0.1:55698)
0618 22:30:56.953579 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:56.953709 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260618 22:30:56.955021 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:56.954596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39701 (local address 127.0.0.1:55700)
0618 22:30:56.954799 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:56.954887 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260618 22:30:57.004891 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 27567
I20260618 22:30:57.017745 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 27213
I20260618 22:30:57.018884 27276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:30:57.132592 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:30:57.131934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:39701 (local address 127.0.0.1:55706)
0618 22:30:57.132128 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0618 22:30:57.132248 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.193:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260618 22:30:57.132747 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 27213
I20260618 22:31:17.150599 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:40485
--webserver_interface=127.10.242.254
--webserver_port=35479
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:40485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:31:17.485584 27743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:17.485932 27743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:17.486064 27743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:17.493450 27743 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260618 22:31:17.493564 27743 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260618 22:31:17.493619 27743 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260618 22:31:17.493686 27743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:31:17.493757 27743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:17.493849 27743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:17.493911 27743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:31:17.493990 27743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:31:17.504105 27743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:40485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:40485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=35479
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:17.505461 27743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:17.506793 27743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:17.518749 27748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:17.518669 27751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:17.518704 27749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:17.519855 27743 server_base.cc:1034] running on GCE node
I20260618 22:31:17.520690 27743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:17.522241 27743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:17.523481 27743 hybrid_clock.cc:648] HybridClock initialized: now 1781821877523401 us; error 56 us; skew 500 ppm
I20260618 22:31:17.523809 27743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:31:17 dist-test-slave-0956 krb5kdc[26662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821877, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:31:17.532265 27743 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:31:17.533819 27743 webserver.cc:460] Webserver started at http://127.10.242.254:35479/ using document root <none> and password file <none>
I20260618 22:31:17.534509 27743 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:17.534600 27743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:17.540088 27743 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:31:17.543006 27758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:17.544461 27743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:31:17.544615 27743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "b8c6ff6457674018b0ccbf5a46d88e4b"
format_stamp: "Formatted at 2026-06-18 22:30:31 on dist-test-slave-0956"
server_key: "7b1f9a1ae2a5b094ee5efbe16fd18959"
server_key_iv: "084e16f71dcf488fb9c884ab19e471cb"
server_key_version: "encryptionkey@0"
I20260618 22:31:17.545673 27743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:17.568604 27743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:17.573047 27743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:17.573313 27743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:17.591156 27743 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:40485
I20260618 22:31:17.591178 27809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:40485 every 8 connection(s)
I20260618 22:31:17.592733 27743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:31:17.593199 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 27743
I20260618 22:31:17.594069 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:39701
--local_ip_for_outbound_sockets=127.10.242.193
--tserver_master_addrs=127.10.242.254:40485
--webserver_port=44709
--webserver_interface=127.10.242.193
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:31:17.599874 27810 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:31:17.603431 27810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: Bootstrap starting.
I20260618 22:31:17.623898 27810 log.cc:824] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:17.640599 27810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:31:17.641196 27810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: Bootstrap complete.
I20260618 22:31:17.650322 27810 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40485 } }
I20260618 22:31:17.651316 27810 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c6ff6457674018b0ccbf5a46d88e4b, State: Initialized, Role: FOLLOWER
I20260618 22:31:17.651966 27810 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40485 } }
I20260618 22:31:17.652179 27810 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:31:17.652299 27810 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:31:17.652462 27810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 2 FOLLOWER]: Advancing to term 3
I20260618 22:31:17.656355 27810 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40485 } }
I20260618 22:31:17.656852 27810 leader_election.cc:302] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8c6ff6457674018b0ccbf5a46d88e4b; no voters:
I20260618 22:31:17.657567 27810 leader_election.cc:288] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260618 22:31:17.657765 27814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 3 FOLLOWER]: Leader election won for term 3
I20260618 22:31:17.658737 27814 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [term 3 LEADER]: Becoming Leader. State: Replica: b8c6ff6457674018b0ccbf5a46d88e4b, State: Running, Role: LEADER
I20260618 22:31:17.659435 27814 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40485 } }
I20260618 22:31:17.659584 27810 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:31:17.665952 27815 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40485 } } }
I20260618 22:31:17.666347 27815 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [sys.catalog]: This master's current role is: LEADER
I20260618 22:31:17.666862 27816 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8c6ff6457674018b0ccbf5a46d88e4b. Latest consensus state: current_term: 3 leader_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c6ff6457674018b0ccbf5a46d88e4b" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40485 } } }
I20260618 22:31:17.667176 27816 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b [sys.catalog]: This master's current role is: LEADER
I20260618 22:31:17.674477 27825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:31:17.683032 27825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:31:17.684604 27825 catalog_manager.cc:1259] Loaded cluster ID: 8e89f0d1bb6e47c68173ab68c89b3fe1
I20260618 22:31:17.684787 27825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:31:17.688975 27825 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:31:17.690907 27825 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: Loaded TSK: 0
I20260618 22:31:17.691179 27825 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b8c6ff6457674018b0ccbf5a46d88e4b: Loaded TSK: 1
F20260618 22:31:17.692416 27825 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f30ed0609ad google::LogMessage::Fail() at ??:0
@ 0x7f30ed06388a google::LogMessage::SendToLog() at ??:0
@ 0x7f30ed0604de google::LogMessage::Flush() at ??:0
@ 0x7f30ed061a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f30f0dc47d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f30f0db373e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f3102a8f1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f3102a8e783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f3102b1a2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f3102b1a0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f3102b645e9 std::function<>::operator()() at ??:0
@ 0x7f3102a92cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f3102a90c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3102b18610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f3102b18439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f30efcac345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f30efcaf91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f30efcaf799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f30efc8aeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f30f2f646db start_thread at ??:0
@ 0x7f30ebefa61f clone at ??:0
W20260618 22:31:17.992058 27812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:17.992369 27812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:17.992472 27812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:17.999974 27812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:18.000092 27812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:18.000207 27812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:31:18.010360 27812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:39701
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=44709
--tserver_master_addrs=127.10.242.254:40485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:18.011639 27812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:18.012925 27812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:18.025887 27858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:18.025885 27859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:18.025933 27861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:18.026845 27812 server_base.cc:1034] running on GCE node
I20260618 22:31:18.027568 27812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:18.028549 27812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:18.029742 27812 hybrid_clock.cc:648] HybridClock initialized: now 1781821878029703 us; error 68 us; skew 500 ppm
I20260618 22:31:18.030050 27812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:31:18 dist-test-slave-0956 krb5kdc[26662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821878, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:31:18.033190 27812 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:31:18.034691 27812 webserver.cc:460] Webserver started at http://127.10.242.193:44709/ using document root <none> and password file <none>
I20260618 22:31:18.035374 27812 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:18.035493 27812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:18.041112 27812 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260618 22:31:18.043924 27868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:18.045375 27812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260618 22:31:18.045549 27812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "7d8b95883e6047f09c698c5f55f90977"
format_stamp: "Formatted at 2026-06-18 22:30:32 on dist-test-slave-0956"
server_key: "32fb911939ca83d16524223987e420b1"
server_key_iv: "48c372ad792a680226d076523c6251e9"
server_key_version: "encryptionkey@0"
I20260618 22:31:18.046603 27812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:18.070804 27812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:18.075058 27812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:18.075305 27812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:18.076326 27812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:18.077824 27812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:18.077924 27812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:18.078035 27812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:18.078114 27812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:18.109599 27812 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:39701
I20260618 22:31:18.109669 27980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:39701 every 8 connection(s)
I20260618 22:31:18.111198 27812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:31:18.116050 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:18.114143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:47053)
0618 22:31:18.114883 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:18.115125 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":281,"threads_started":1}
W20260618 22:31:18.116694 27981 heartbeater.cc:644] Failed to heartbeat to 127.10.242.254:40485 (0 consecutive failures): Network error: Failed to ping master at 127.10.242.254:40485: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
I20260618 22:31:18.117605 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:18.117217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:34299)
0618 22:31:18.117368 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:18.117470 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260618 22:31:18.117866 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 27812
I20260618 22:31:18.118538 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:18.118143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:42983)
0618 22:31:18.118295 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:18.118386 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260618 22:31:18.118598 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:43839
--local_ip_for_outbound_sockets=127.10.242.194
--tserver_master_addrs=127.10.242.254:40485
--webserver_port=39637
--webserver_interface=127.10.242.194
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:31:18.118897 27981 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260618 22:31:18.470112 27984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:18.470425 27984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:18.470521 27984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:18.478022 27984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:18.478156 27984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:18.478279 27984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:31:18.488586 27984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:43839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=39637
--tserver_master_addrs=127.10.242.254:40485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:18.490007 27984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:18.491339 27984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:18.505621 27989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:18.505817 27990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:18.505605 27992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:18.506404 27984 server_base.cc:1034] running on GCE node
I20260618 22:31:18.507120 27984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:18.508155 27984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:18.509374 27984 hybrid_clock.cc:648] HybridClock initialized: now 1781821878509318 us; error 48 us; skew 500 ppm
I20260618 22:31:18.509701 27984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:31:18 dist-test-slave-0956 krb5kdc[26662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821878, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:31:18.512897 27984 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:31:18.514442 27984 webserver.cc:460] Webserver started at http://127.10.242.194:39637/ using document root <none> and password file <none>
I20260618 22:31:18.515152 27984 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:18.515280 27984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:18.521231 27984 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260618 22:31:18.524432 27999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:18.526157 27984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:31:18.526356 27984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "dc19bc4ce9844a68a5401b9ee888f302"
format_stamp: "Formatted at 2026-06-18 22:30:32 on dist-test-slave-0956"
server_key: "5501cc29c30685023820244dd6eb1af2"
server_key_iv: "34ca8ff20a138913d0d6084b8faa89d7"
server_key_version: "encryptionkey@0"
I20260618 22:31:18.527395 27984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:18.544543 27984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:18.549532 27984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:18.549815 27984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:18.550879 27984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:18.552307 27984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:18.552420 27984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:18.552528 27984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:18.552603 27984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:18.584472 27984 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:43839
I20260618 22:31:18.584520 28111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:43839 every 8 connection(s)
I20260618 22:31:18.585947 27984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:31:18.590628 28114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:18.588756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:39329)
0618 22:31:18.589553 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:18.589768 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":327,"threads_started":1}
I20260618 22:31:18.591243 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 27984
W20260618 22:31:18.591259 28112 heartbeater.cc:644] Failed to heartbeat to 127.10.242.254:40485 (0 consecutive failures): Network error: Failed to ping master at 127.10.242.254:40485: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
I20260618 22:31:18.591933 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:43709
--local_ip_for_outbound_sockets=127.10.242.195
--tserver_master_addrs=127.10.242.254:40485
--webserver_port=41263
--webserver_interface=127.10.242.195
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:31:18.592213 28114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:18.591839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:40457)
0618 22:31:18.591995 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:18.592073 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260618 22:31:18.593040 28114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:18.592755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:33437)
0618 22:31:18.592869 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:18.592938 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260618 22:31:18.593310 28112 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260618 22:31:18.916887 28115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:18.917218 28115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:18.917323 28115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:18.924580 28115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:18.924705 28115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:18.924820 28115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:31:18.934681 28115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:43709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=41263
--tserver_master_addrs=127.10.242.254:40485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:18.935916 28115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:18.937253 28115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:18.950845 28120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:18.950939 28121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:18.950910 28123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:18.951902 28115 server_base.cc:1034] running on GCE node
I20260618 22:31:18.952548 28115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:18.953610 28115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:18.954844 28115 hybrid_clock.cc:648] HybridClock initialized: now 1781821878954786 us; error 56 us; skew 500 ppm
I20260618 22:31:18.955211 28115 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:31:18 dist-test-slave-0956 krb5kdc[26662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821878, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:31:18.958696 28115 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:31:18.960381 28115 webserver.cc:460] Webserver started at http://127.10.242.195:41263/ using document root <none> and password file <none>
I20260618 22:31:18.961199 28115 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:18.961333 28115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:18.967279 28115 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260618 22:31:18.970484 28130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:18.972152 28115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:31:18.972365 28115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "d0f3eb64d664474cbf8679f78a965f2b"
format_stamp: "Formatted at 2026-06-18 22:30:33 on dist-test-slave-0956"
server_key: "90dd29a27cdf9e8e36d4506069f9590a"
server_key_iv: "b4aef2d90f23d9b1e91055358e7a127c"
server_key_version: "encryptionkey@0"
I20260618 22:31:18.973508 28115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:18.987939 28115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:18.992349 28115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:18.992612 28115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:18.993680 28115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:18.995077 28115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:18.995183 28115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:18.995316 28115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:18.995388 28115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:19.026691 28115 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:43709
I20260618 22:31:19.026758 28242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:43709 every 8 connection(s)
I20260618 22:31:19.028146 28115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:31:19.033164 28245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:19.031150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:53525)
0618 22:31:19.031922 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:19.032180 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":303,"threads_started":1}
W20260618 22:31:19.033815 28243 heartbeater.cc:644] Failed to heartbeat to 127.10.242.254:40485 (0 consecutive failures): Network error: Failed to ping master at 127.10.242.254:40485: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
I20260618 22:31:19.034786 28245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:19.034346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:57237)
0618 22:31:19.034552 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:19.034642 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260618 22:31:19.035022 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 28115
I20260618 22:31:19.035749 28245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:19.035407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:37197)
0618 22:31:19.035560 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:19.035637 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260618 22:31:19.035949 28243 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260618 22:31:19.120433 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:19.119567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:53233)
0618 22:31:19.120084 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:19.120188 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":248,"threads_started":1}
I20260618 22:31:19.594767 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:19.593860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:39347)
0618 22:31:19.594397 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:19.594516 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":247,"threads_started":1}
I20260618 22:31:20.037554 28249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:20.036581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:47589)
0618 22:31:20.037073 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:20.037256 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":240,"threads_started":1}
I20260618 22:31:20.122399 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:20.121576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:45557)
0618 22:31:20.122125 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:20.122215 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":260,"threads_started":1}
I20260618 22:31:20.596616 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:20.595716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:37495)
0618 22:31:20.596249 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:20.596357 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":243,"threads_started":1}
I20260618 22:31:21.039340 28252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:21.038404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:38491)
0618 22:31:21.038962 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:21.039076 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":254,"threads_started":1}
I20260618 22:31:21.124136 28253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:21.123307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:43199)
0618 22:31:21.123840 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:21.123928 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":278,"threads_started":1}
I20260618 22:31:21.598459 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:21.597652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:45589)
0618 22:31:21.598152 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:21.598248 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":238,"threads_started":1}
I20260618 22:31:22.041206 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:22.040342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:34269)
0618 22:31:22.040857 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:22.040961 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":291,"threads_started":1}
I20260618 22:31:22.126084 28256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:22.125214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:39785)
0618 22:31:22.125752 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:22.125871 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":274,"threads_started":1}
I20260618 22:31:22.600168 28257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:22.599340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:51371)
0618 22:31:22.599868 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:22.599958 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":252,"threads_started":1}
I20260618 22:31:23.042847 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:23.042074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:48193)
0618 22:31:23.042567 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:23.042650 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":231,"threads_started":1}
I20260618 22:31:23.127792 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:23.127036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:43397)
0618 22:31:23.127530 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:23.127620 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":233,"threads_started":1}
I20260618 22:31:23.602222 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:23.601277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:42059)
0618 22:31:23.601910 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:23.602021 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":300,"threads_started":1}
I20260618 22:31:24.044479 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:24.043756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:52599)
0618 22:31:24.044231 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:24.044313 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":250,"threads_started":1}
I20260618 22:31:24.129484 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:24.128680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:49783)
0618 22:31:24.129186 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:24.129303 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":227,"threads_started":1}
I20260618 22:31:24.604020 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:24.603255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:53013)
0618 22:31:24.603762 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:24.603857 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":260,"threads_started":1}
I20260618 22:31:25.046233 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:25.045485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:40851)
0618 22:31:25.045962 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:25.046068 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":237,"threads_started":1}
I20260618 22:31:25.131273 28265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:25.130473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:38827)
0618 22:31:25.130943 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:25.131027 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":199,"threads_started":1}
I20260618 22:31:25.606014 28266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:25.605147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:37031)
0618 22:31:25.605706 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:25.605797 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":234,"threads_started":1}
I20260618 22:31:26.048425 28267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:26.047524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:54289)
0618 22:31:26.048110 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:26.048204 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":260,"threads_started":1}
I20260618 22:31:26.133208 28268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:26.132306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:37485)
0618 22:31:26.132821 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:26.132929 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":272,"threads_started":1}
I20260618 22:31:26.607834 28269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:26.607025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:42997)
0618 22:31:26.607524 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:26.607657 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":268,"threads_started":1}
I20260618 22:31:27.050318 28270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:27.049428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:34319)
0618 22:31:27.049993 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:27.050121 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":306,"threads_started":1}
I20260618 22:31:27.134996 28271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:27.134120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:41025)
0618 22:31:27.134708 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:27.134803 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":257,"threads_started":1}
I20260618 22:31:27.609463 28272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:27.608696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:43827)
0618 22:31:27.609212 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:27.609292 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":249,"threads_started":1}
I20260618 22:31:28.051932 28273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:28.051162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:35023)
0618 22:31:28.051667 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:28.051751 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":213,"threads_started":1}
I20260618 22:31:28.136773 28274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:28.135975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:44941)
0618 22:31:28.136481 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:28.136604 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":279,"threads_started":1}
I20260618 22:31:28.611327 28275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:28.610522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:60745)
0618 22:31:28.611059 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:28.611160 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":264,"threads_started":1}
I20260618 22:31:29.053668 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:29.052853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:52535)
0618 22:31:29.053393 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:29.053490 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":302,"threads_started":1}
I20260618 22:31:29.138497 28277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:29.137758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:49413)
0618 22:31:29.138211 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:29.138313 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":211,"threads_started":1}
I20260618 22:31:29.613039 28278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:29.612216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:36773)
0618 22:31:29.612752 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:29.612857 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":262,"threads_started":1}
I20260618 22:31:30.055442 28279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:30.054683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:42539)
0618 22:31:30.055163 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:30.055262 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":232,"threads_started":1}
I20260618 22:31:30.140282 28280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:30.139528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:43601)
0618 22:31:30.140021 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:30.140111 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":242,"threads_started":1}
I20260618 22:31:30.615003 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:30.614211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:57005)
0618 22:31:30.614755 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:30.614846 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":272,"threads_started":1}
I20260618 22:31:31.057461 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:31.056638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:45341)
0618 22:31:31.057178 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:31.057266 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":250,"threads_started":1}
I20260618 22:31:31.142208 28283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:31.141367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:45781)
0618 22:31:31.141904 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:31.142016 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":236,"threads_started":1}
I20260618 22:31:31.616905 28284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:31.616028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:57701)
0618 22:31:31.616555 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:31.616724 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":284,"threads_started":1}
I20260618 22:31:32.059348 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:32.058499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:36265)
0618 22:31:32.059049 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:32.059149 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":279,"threads_started":1}
I20260618 22:31:32.143922 28286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:32.143116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:56001)
0618 22:31:32.143596 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:32.143738 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":232,"threads_started":1}
I20260618 22:31:32.618649 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:32.617906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:59603)
0618 22:31:32.618372 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:32.618461 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":219,"threads_started":1}
I20260618 22:31:33.061149 28288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:33.060311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:53897)
0618 22:31:33.060816 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:33.060917 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":227,"threads_started":1}
I20260618 22:31:33.145946 28289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:33.145032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.193:58377)
0618 22:31:33.145568 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:33.145750 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":288,"threads_started":1}
I20260618 22:31:33.620643 28290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:33.619841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.194:39753)
0618 22:31:33.620340 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:33.620449 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":242,"threads_started":1}
I20260618 22:31:34.036809 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 27812
I20260618 22:31:34.055500 28291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:34.037913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58406 (local address 127.10.242.193:39701)
0618 22:31:34.038500 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:34.038519 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:34.038829 (+ 310us) server_negotiation.cc:407] Connection header received
0618 22:31:34.040573 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:34.040583 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:34.041026 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:34.041394 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:34.042915 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:34.044132 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:34.045446 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:34.046139 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:34.046314 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:34.048885 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:34.048893 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:34.048911 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:34.048952 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:34.052065 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:34.052804 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:34.052811 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:34.052819 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:34.053028 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:34.053405 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:34.053410 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:34.053414 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:34.053987 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:34.054261 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:34.054778 (+ 517us) server_negotiation.cc:299] Negotiation successful
0618 22:31:34.055218 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":552,"threads_started":1}
I20260618 22:31:34.056954 27976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:34.062949 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:34.062248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:40485 (local address 127.10.242.195:41777)
0618 22:31:34.062685 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:34.062779 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.254:40485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":232,"threads_started":1}
I20260618 22:31:34.141346 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 27812
I20260618 22:31:34.155805 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 27984
I20260618 22:31:34.173974 28294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:34.156715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53782 (local address 127.10.242.194:43839)
0618 22:31:34.157265 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:34.157279 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:34.157316 (+ 37us) server_negotiation.cc:407] Connection header received
0618 22:31:34.158801 (+ 1485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:34.158811 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:34.159248 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:34.159549 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:34.161044 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:34.162418 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:34.163846 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:34.164603 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:34.164770 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:34.167639 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:34.167650 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:34.167669 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:34.167776 (+ 107us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:34.170875 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:34.171594 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:34.171602 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:34.171608 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:34.171798 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:34.172115 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:34.172118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:34.172120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:34.172649 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:34.172919 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:34.173300 (+ 381us) server_negotiation.cc:299] Negotiation successful
0618 22:31:34.173682 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":260,"threads_started":1}
I20260618 22:31:34.175326 28107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:34.257629 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 27984
I20260618 22:31:34.271992 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 28115
I20260618 22:31:34.290136 28296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:34.272910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55526 (local address 127.10.242.195:43709)
0618 22:31:34.273472 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:34.273484 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:34.273524 (+ 40us) server_negotiation.cc:407] Connection header received
0618 22:31:34.275165 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:34.275176 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:34.275629 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:34.275956 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:34.277438 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:34.278704 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:34.280091 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:34.280949 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:34.281162 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:34.283795 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:34.283804 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:34.283820 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:34.283863 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:34.286775 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:34.287494 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:34.287501 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:34.287508 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:34.287685 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:34.288057 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:34.288061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:34.288063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:34.288621 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:34.288907 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:34.289367 (+ 460us) server_negotiation.cc:299] Negotiation successful
0618 22:31:34.289831 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":264,"threads_started":1}
I20260618 22:31:34.291617 28238 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:34.374075 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 28115
2026-06-18T22:31:34Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64142 ms)
[----------] 30 tests from SecurityITest (289960 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:31:34 dist-test-slave-0956 krb5kdc[28300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:31:34 dist-test-slave-0956 krb5kdc[28300](info): set up 2 sockets
Jun 18 22:31:34 dist-test-slave-0956 krb5kdc[28300](info): commencing operation
krb5kdc: starting...
W20260618 22:31:35.388891 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.005s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:31:35 dist-test-slave-0956 krb5kdc[28300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:31:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:31:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:31:35.625465 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:39257
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:44997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:39257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:31:35.951822 28313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:35.952171 28313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:35.952304 28313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:35.959743 28313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:31:35.959885 28313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:35.959973 28313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:35.960052 28313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:31:35.960112 28313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:31:35.970240 28313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:44997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:39257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:39257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:35.971637 28313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:35.972994 28313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:35.985169 28318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:35.985116 28319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:35.985179 28321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:35.985489 28313 server_base.cc:1034] running on GCE node
I20260618 22:31:35.986495 28313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:35.987985 28313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:35.989223 28313 hybrid_clock.cc:648] HybridClock initialized: now 1781821895989148 us; error 45 us; skew 500 ppm
I20260618 22:31:35.989617 28313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:31:35 dist-test-slave-0956 krb5kdc[28300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821895, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:31:35.997593 28313 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:31:35.999220 28313 webserver.cc:460] Webserver started at http://127.10.242.254:44735/ using document root <none> and password file <none>
I20260618 22:31:35.999969 28313 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:36.000085 28313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:36.000370 28313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:36.003657 28313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "1bfc9f90692b4babb1956ffbab6204a3"
format_stamp: "Formatted at 2026-06-18 22:31:35 on dist-test-slave-0956"
server_key: "2d34d0fd84796eb7371187aa7fe2001e"
server_key_iv: "f90e6d9d959129466ba699846a08f587"
server_key_version: "encryptionkey@0"
I20260618 22:31:36.004468 28313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "1bfc9f90692b4babb1956ffbab6204a3"
format_stamp: "Formatted at 2026-06-18 22:31:35 on dist-test-slave-0956"
server_key: "2d34d0fd84796eb7371187aa7fe2001e"
server_key_iv: "f90e6d9d959129466ba699846a08f587"
server_key_version: "encryptionkey@0"
I20260618 22:31:36.010490 28313 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260618 22:31:36.014237 28328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:36.015676 28313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:31:36.015846 28313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "1bfc9f90692b4babb1956ffbab6204a3"
format_stamp: "Formatted at 2026-06-18 22:31:35 on dist-test-slave-0956"
server_key: "2d34d0fd84796eb7371187aa7fe2001e"
server_key_iv: "f90e6d9d959129466ba699846a08f587"
server_key_version: "encryptionkey@0"
I20260618 22:31:36.016043 28313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:36.033179 28313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:36.037315 28313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:36.037582 28313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:36.055109 28313 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:39257
I20260618 22:31:36.055127 28379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:39257 every 8 connection(s)
I20260618 22:31:36.056486 28313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:31:36.058544 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 28313
I20260618 22:31:36.058744 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:31:36.059361 11211 external_mini_cluster.cc:1428] Setting key 071efad7ae53449d1d3bad8055c82a34
I20260618 22:31:36.060485 28380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 18 22:31:36 dist-test-slave-0956 krb5kdc[28300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:31:36.074220 28380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3: Bootstrap starting.
I20260618 22:31:36.078059 28380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:36.079397 28380 log.cc:824] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:36.082966 28380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3: No bootstrap required, opened a new log
I20260618 22:31:36.083913 28383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:36.061547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35274 (local address 127.10.242.254:39257)
0618 22:31:36.062351 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:36.062366 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:36.062415 (+ 49us) server_negotiation.cc:407] Connection header received
0618 22:31:36.064089 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:36.064117 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:36.064661 (+ 544us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:36.065143 (+ 482us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:36.066612 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:36.068608 (+ 1996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:36.070061 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:36.071081 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:36.071281 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:36.075289 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:36.075301 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:36.075321 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:36.075366 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:36.079175 (+ 3809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:36.080001 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:36.080010 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:36.080022 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:36.080198 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:36.080606 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:36.080610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:36.080612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:36.081386 (+ 774us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:36.081669 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:36.082058 (+ 389us) server_negotiation.cc:299] Negotiation successful
0618 22:31:36.082516 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":345,"threads_started":1}
I20260618 22:31:36.091291 28380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfc9f90692b4babb1956ffbab6204a3" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39257 } }
I20260618 22:31:36.091681 28380 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:36.091781 28380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bfc9f90692b4babb1956ffbab6204a3, State: Initialized, Role: FOLLOWER
I20260618 22:31:36.092332 28380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfc9f90692b4babb1956ffbab6204a3" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39257 } }
I20260618 22:31:36.092547 28380 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:31:36.092674 28380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:31:36.092849 28380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:36.095115 28380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfc9f90692b4babb1956ffbab6204a3" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39257 } }
I20260618 22:31:36.095613 28380 leader_election.cc:302] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1bfc9f90692b4babb1956ffbab6204a3; no voters:
I20260618 22:31:36.096210 28380 leader_election.cc:288] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:31:36.096349 28385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:31:36.097364 28385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3 [term 1 LEADER]: Becoming Leader. State: Replica: 1bfc9f90692b4babb1956ffbab6204a3, State: Running, Role: LEADER
I20260618 22:31:36.098161 28385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfc9f90692b4babb1956ffbab6204a3" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39257 } }
I20260618 22:31:36.098302 28380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:31:36.100728 28387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1bfc9f90692b4babb1956ffbab6204a3. Latest consensus state: current_term: 1 leader_uuid: "1bfc9f90692b4babb1956ffbab6204a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfc9f90692b4babb1956ffbab6204a3" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39257 } } }
I20260618 22:31:36.100703 28386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1bfc9f90692b4babb1956ffbab6204a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfc9f90692b4babb1956ffbab6204a3" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39257 } } }
I20260618 22:31:36.101179 28387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3 [sys.catalog]: This master's current role is: LEADER
I20260618 22:31:36.101179 28386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3 [sys.catalog]: This master's current role is: LEADER
I20260618 22:31:36.102180 28392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:31:36.107333 28392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:31:36.118675 28392 catalog_manager.cc:1347] Generated new cluster ID: 8cff6d2783b04b5da064b5e3169ee71a
I20260618 22:31:36.118798 28392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:31:36.127797 28392 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:31:36.129714 28392 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:31:36.140944 28392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3: Generated new TSK 0
I20260618 22:31:36.141860 28392 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
I20260618 22:31:36.210530 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:39257
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:44997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:31:36.535879 28406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:36.536180 28406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:36.536320 28406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:36.544107 28406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:36.544250 28406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:36.544409 28406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:31:36.554428 28406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:44997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:39257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:36.555748 28406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:36.557166 28406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:36.569653 28411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:36.569625 28412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:36.569697 28414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:36.570542 28406 server_base.cc:1034] running on GCE node
I20260618 22:31:36.571245 28406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:36.572297 28406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:36.573522 28406 hybrid_clock.cc:648] HybridClock initialized: now 1781821896573469 us; error 55 us; skew 500 ppm
I20260618 22:31:36.573843 28406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:31:36 dist-test-slave-0956 krb5kdc[28300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821896, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:31:36.577024 28406 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:31:36.578511 28406 webserver.cc:460] Webserver started at http://127.10.242.193:36889/ using document root <none> and password file <none>
I20260618 22:31:36.579196 28406 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:36.579311 28406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:36.579610 28406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:36.582679 28406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "730f147613b94fab881b09c2eb640d38"
format_stamp: "Formatted at 2026-06-18 22:31:36 on dist-test-slave-0956"
server_key: "c06be96406308b5fa187b996621ffd73"
server_key_iv: "55330992bc123a62915b2912cf49fdde"
server_key_version: "encryptionkey@0"
I20260618 22:31:36.583434 28406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "730f147613b94fab881b09c2eb640d38"
format_stamp: "Formatted at 2026-06-18 22:31:36 on dist-test-slave-0956"
server_key: "c06be96406308b5fa187b996621ffd73"
server_key_iv: "55330992bc123a62915b2912cf49fdde"
server_key_version: "encryptionkey@0"
I20260618 22:31:36.588670 28406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260618 22:31:36.592142 28421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:36.593634 28406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:31:36.593814 28406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "730f147613b94fab881b09c2eb640d38"
format_stamp: "Formatted at 2026-06-18 22:31:36 on dist-test-slave-0956"
server_key: "c06be96406308b5fa187b996621ffd73"
server_key_iv: "55330992bc123a62915b2912cf49fdde"
server_key_version: "encryptionkey@0"
I20260618 22:31:36.594009 28406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:36.623548 28406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:36.627786 28406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:36.628059 28406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:36.629051 28406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:36.630448 28406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:36.630518 28406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:36.630626 28406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:36.630690 28406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:36.660683 28406 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:35775
I20260618 22:31:36.660727 28533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:35775 every 8 connection(s)
I20260618 22:31:36.662027 28406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:31:36.666620 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 28406
I20260618 22:31:36.666839 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:31:36.667627 11211 external_mini_cluster.cc:1428] Setting key ea41c34e2c1aa1758bad93bc4835d759
Jun 18 22:31:36 dist-test-slave-0956 krb5kdc[28300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821896, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:31:36.684646 28536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:36.664606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:58571 (local address 127.10.242.254:39257)
0618 22:31:36.665074 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:36.665102 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:36.666115 (+ 1013us) server_negotiation.cc:407] Connection header received
0618 22:31:36.668574 (+ 2459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:36.668579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:36.668669 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:36.668775 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:36.670684 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:36.671405 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:36.673521 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:36.673951 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:36.674052 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:36.678242 (+ 4190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:36.678248 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:36.678253 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:36.678300 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:36.681371 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:36.682238 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:36.682243 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:36.682247 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:36.682396 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:36.682811 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:36.682815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:36.682818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:36.683142 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:36.683274 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:36.684085 (+ 811us) server_negotiation.cc:299] Negotiation successful
0618 22:31:36.684304 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":215,"threads_started":1}
I20260618 22:31:36.685598 28537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:36.665196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:39257 (local address 127.10.242.193:58571)
0618 22:31:36.665948 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:36.665987 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:36.668296 (+ 2309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:36.669023 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:36.669032 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:36.669648 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:31:36.670494 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:36.670520 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:36.671553 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:36.671558 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:36.673363 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:36.673378 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:36.674175 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:36.674181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:36.674469 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:36.675204 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:31:36.675242 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:31:36.677949 (+ 2707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:31:36.681561 (+ 3612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:31:36.681570 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:31:36.681586 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:31:36.682106 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:31:36.682507 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:31:36.682510 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:31:36.682513 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:31:36.682713 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:31:36.683294 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:31:36.683302 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:31:36.683678 (+ 376us) client_negotiation.cc:769] Sending connection context
0618 22:31:36.683942 (+ 264us) client_negotiation.cc:240] Negotiation successful
0618 22:31:36.684390 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":292,"threads_started":1}
I20260618 22:31:36.687162 28534 heartbeater.cc:347] Connected to a master server at 127.10.242.254:39257
I20260618 22:31:36.687489 28534 heartbeater.cc:464] Registering TS with master...
I20260618 22:31:36.688339 28534 heartbeater.cc:511] Master 127.10.242.254:39257 requested a full tablet report, sending...
I20260618 22:31:36.690642 28345 ts_manager.cc:194] Registered new tserver with Master: 730f147613b94fab881b09c2eb640d38 (127.10.242.193:35775)
I20260618 22:31:36.692118 28345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:58571
WARNING: no policy specified for kudu/127.10.242.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
I20260618 22:31:36.723784 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:39257
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:44997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:31:37.047814 28540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:37.048110 28540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:37.048230 28540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:37.055841 28540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:37.055981 28540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:37.056140 28540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:31:37.066522 28540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:44997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:39257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:37.067777 28540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:37.069049 28540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:37.082119 28545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:37.082098 28548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:37.082065 28546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:37.082903 28540 server_base.cc:1034] running on GCE node
I20260618 22:31:37.083514 28540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:37.084486 28540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:37.085708 28540 hybrid_clock.cc:648] HybridClock initialized: now 1781821897085648 us; error 50 us; skew 500 ppm
I20260618 22:31:37.086041 28540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:31:37 dist-test-slave-0956 krb5kdc[28300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821897, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:31:37.089254 28540 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:31:37.090757 28540 webserver.cc:460] Webserver started at http://127.10.242.194:46461/ using document root <none> and password file <none>
I20260618 22:31:37.091444 28540 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:37.091588 28540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:37.091889 28540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:37.094869 28540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "942e5af2098d4453a190a0a97834ae29"
format_stamp: "Formatted at 2026-06-18 22:31:37 on dist-test-slave-0956"
server_key: "093acc1b4c15698505c5d3ce62d27dd2"
server_key_iv: "63a1c6609f22731bb946ef0dd43c9f92"
server_key_version: "encryptionkey@0"
I20260618 22:31:37.095667 28540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "942e5af2098d4453a190a0a97834ae29"
format_stamp: "Formatted at 2026-06-18 22:31:37 on dist-test-slave-0956"
server_key: "093acc1b4c15698505c5d3ce62d27dd2"
server_key_iv: "63a1c6609f22731bb946ef0dd43c9f92"
server_key_version: "encryptionkey@0"
I20260618 22:31:37.101204 28540 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260618 22:31:37.105363 28555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:37.106916 28540 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260618 22:31:37.107121 28540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "942e5af2098d4453a190a0a97834ae29"
format_stamp: "Formatted at 2026-06-18 22:31:37 on dist-test-slave-0956"
server_key: "093acc1b4c15698505c5d3ce62d27dd2"
server_key_iv: "63a1c6609f22731bb946ef0dd43c9f92"
server_key_version: "encryptionkey@0"
I20260618 22:31:37.107342 28540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:37.123142 28540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:37.127404 28540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:37.127696 28540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:37.128690 28540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:37.130087 28540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:37.130164 28540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:37.130280 28540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:37.130343 28540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:37.161213 28540 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:40957
I20260618 22:31:37.161249 28667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:40957 every 8 connection(s)
I20260618 22:31:37.162541 28540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:31:37.166301 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 28540
I20260618 22:31:37.166476 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:31:37.166999 11211 external_mini_cluster.cc:1428] Setting key 2310e631663f43af2feff9e448f857f8
Jun 18 22:31:37 dist-test-slave-0956 krb5kdc[28300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821897, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:31:37.182700 28536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:37.165131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:44149 (local address 127.10.242.254:39257)
0618 22:31:37.165315 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:37.165320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:37.166667 (+ 1347us) server_negotiation.cc:407] Connection header received
0618 22:31:37.168664 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:37.168670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:37.168753 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:37.168845 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:37.170454 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.171174 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.172752 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.173241 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.173359 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:37.176734 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:37.176739 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:37.176742 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:37.176777 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:37.179346 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:37.180053 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:37.180058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:37.180060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:37.180182 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:37.180587 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:37.180592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:37.180595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:37.180927 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:37.181103 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:37.182266 (+ 1163us) server_negotiation.cc:299] Negotiation successful
0618 22:31:37.182450 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260618 22:31:37.184387 28670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:37.165702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:39257 (local address 127.10.242.194:44149)
0618 22:31:37.166484 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:37.166533 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:37.168389 (+ 1856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:37.169125 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:37.169137 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:37.169667 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:31:37.170263 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:37.170282 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.171341 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.171345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:37.172605 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:37.172618 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.173478 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.173486 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:37.173724 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:37.174382 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:31:37.174405 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:31:37.176510 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:31:37.179482 (+ 2972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:31:37.179491 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:31:37.179509 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:31:37.179941 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:31:37.180281 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:31:37.180286 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:31:37.180290 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:31:37.180484 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:31:37.181405 (+ 921us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:31:37.181413 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:31:37.181864 (+ 451us) client_negotiation.cc:769] Sending connection context
0618 22:31:37.182179 (+ 315us) client_negotiation.cc:240] Negotiation successful
0618 22:31:37.182694 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":301,"threads_started":1}
I20260618 22:31:37.185978 28668 heartbeater.cc:347] Connected to a master server at 127.10.242.254:39257
I20260618 22:31:37.186376 28668 heartbeater.cc:464] Registering TS with master...
I20260618 22:31:37.187367 28668 heartbeater.cc:511] Master 127.10.242.254:39257 requested a full tablet report, sending...
I20260618 22:31:37.189038 28345 ts_manager.cc:194] Registered new tserver with Master: 942e5af2098d4453a190a0a97834ae29 (127.10.242.194:40957)
WARNING: no policy specified for kudu/127.10.242.195@KRBTEST.COM; defaulting to no policy
I20260618 22:31:37.189890 28345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:44149
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
I20260618 22:31:37.220037 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:39257
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:44997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:31:37.547214 28673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:37.547511 28673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:37.547605 28673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:37.554972 28673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:37.555099 28673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:37.555213 28673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:31:37.565062 28673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:44997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:39257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:37.566519 28673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:37.567811 28673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:37.580574 28678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:37.580650 28681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:37.580605 28679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:37.581581 28673 server_base.cc:1034] running on GCE node
I20260618 22:31:37.582211 28673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:37.583153 28673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:37.584357 28673 hybrid_clock.cc:648] HybridClock initialized: now 1781821897584307 us; error 45 us; skew 500 ppm
I20260618 22:31:37.584681 28673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:31:37 dist-test-slave-0956 krb5kdc[28300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821897, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:31:37.587664 28673 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:31:37.589023 28673 webserver.cc:460] Webserver started at http://127.10.242.195:46385/ using document root <none> and password file <none>
I20260618 22:31:37.589711 28673 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:37.589809 28673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:37.590060 28673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:37.592905 28673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "8ab3bf4bd234458eb721ca5f93a54794"
format_stamp: "Formatted at 2026-06-18 22:31:37 on dist-test-slave-0956"
server_key: "306570d6a2082a7606389b519f09574e"
server_key_iv: "1bf2ac9703aa822b9b7ea64efd752452"
server_key_version: "encryptionkey@0"
I20260618 22:31:37.593672 28673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "8ab3bf4bd234458eb721ca5f93a54794"
format_stamp: "Formatted at 2026-06-18 22:31:37 on dist-test-slave-0956"
server_key: "306570d6a2082a7606389b519f09574e"
server_key_iv: "1bf2ac9703aa822b9b7ea64efd752452"
server_key_version: "encryptionkey@0"
I20260618 22:31:37.599251 28673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260618 22:31:37.603089 28688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:37.604586 28673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:31:37.604743 28673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "8ab3bf4bd234458eb721ca5f93a54794"
format_stamp: "Formatted at 2026-06-18 22:31:37 on dist-test-slave-0956"
server_key: "306570d6a2082a7606389b519f09574e"
server_key_iv: "1bf2ac9703aa822b9b7ea64efd752452"
server_key_version: "encryptionkey@0"
I20260618 22:31:37.604934 28673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:37.618160 28673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:37.622656 28673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:37.622939 28673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:37.623929 28673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:37.625344 28673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:37.625427 28673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:37.625527 28673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:37.625583 28673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:37.656558 28673 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:33399
I20260618 22:31:37.656598 28800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:33399 every 8 connection(s)
I20260618 22:31:37.657874 28673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:31:37.662953 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 28673
I20260618 22:31:37.663129 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:31:37.663590 11211 external_mini_cluster.cc:1428] Setting key 1a4f5afc8822005c2c12b17bb5237d64
Jun 18 22:31:37 dist-test-slave-0956 krb5kdc[28300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821897, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:31:37.679474 28536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:37.660437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:45225 (local address 127.10.242.254:39257)
0618 22:31:37.660636 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:37.660643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:37.661956 (+ 1313us) server_negotiation.cc:407] Connection header received
0618 22:31:37.664625 (+ 2669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:37.664629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:37.664709 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:37.664803 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:37.666907 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.667635 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.669296 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.669719 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.669823 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:37.673358 (+ 3535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:37.673363 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:37.673366 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:37.673401 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:37.676024 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:37.676830 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:37.676834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:37.676836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:37.676971 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:37.677426 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:37.677431 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:37.677433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:37.677777 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:37.677917 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:37.678912 (+ 995us) server_negotiation.cc:299] Negotiation successful
0618 22:31:37.679185 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260618 22:31:37.680411 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:37.660942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:39257 (local address 127.10.242.195:45225)
0618 22:31:37.661760 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:37.661829 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:37.664102 (+ 2273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:37.665119 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:37.665131 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:37.665879 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:31:37.666667 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:37.666693 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.667802 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.667807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:37.669142 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:37.669158 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.669931 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.669937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:37.670144 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:37.670777 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:31:37.670803 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:31:37.673120 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:31:37.676192 (+ 3072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:31:37.676202 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:31:37.676220 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:31:37.676681 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:31:37.677104 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:31:37.677109 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:31:37.677113 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:31:37.677309 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:31:37.677926 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:31:37.677934 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:31:37.678436 (+ 502us) client_negotiation.cc:769] Sending connection context
0618 22:31:37.678769 (+ 333us) client_negotiation.cc:240] Negotiation successful
0618 22:31:37.679272 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":343,"threads_started":1}
I20260618 22:31:37.681933 28801 heartbeater.cc:347] Connected to a master server at 127.10.242.254:39257
I20260618 22:31:37.682257 28801 heartbeater.cc:464] Registering TS with master...
I20260618 22:31:37.683004 28801 heartbeater.cc:511] Master 127.10.242.254:39257 requested a full tablet report, sending...
I20260618 22:31:37.684456 28345 ts_manager.cc:194] Registered new tserver with Master: 8ab3bf4bd234458eb721ca5f93a54794 (127.10.242.195:33399)
I20260618 22:31:37.685328 28345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:45225
I20260618 22:31:37.692057 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:31:37.695255 28534 heartbeater.cc:503] Master 127.10.242.254:39257 was elected leader, sending a full tablet report...
I20260618 22:31:37.710040 28536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:37.696784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35280 (local address 127.10.242.254:39257)
0618 22:31:37.696969 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:37.696975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:37.697505 (+ 530us) server_negotiation.cc:407] Connection header received
0618 22:31:37.697635 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:37.697638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:37.697699 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:37.697807 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:37.699373 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.700131 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.701771 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.702202 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.702307 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:37.704636 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:37.704641 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:37.704644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:37.704680 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:37.707255 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:37.707928 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:37.707932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:37.707935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:37.708108 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:37.708575 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:37.708579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:37.708581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:37.708921 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:37.709141 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:37.709545 (+ 404us) server_negotiation.cc:299] Negotiation successful
0618 22:31:37.709759 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:31:37.714483 28345 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:31:37.722043 28345 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:31:37.757434 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:37.735231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:33399 (local address 127.0.0.1:57450)
0618 22:31:37.747292 (+ 12061us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:37.747343 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:37.747559 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:37.749447 (+ 1888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:37.749453 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:37.749490 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:37.750032 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:37.750045 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.752167 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.752173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:37.754501 (+ 2328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:37.754511 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.756348 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.756355 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:37.756601 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:37.756666 (+ 65us) client_negotiation.cc:769] Sending connection context
0618 22:31:37.756794 (+ 128us) client_negotiation.cc:240] Negotiation successful
0618 22:31:37.756989 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11792,"thread_start_us":11634,"threads_started":1}
I20260618 22:31:37.757465 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:37.735899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57450 (local address 127.10.242.195:33399)
0618 22:31:37.746809 (+ 10910us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:37.746819 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:37.748415 (+ 1596us) server_negotiation.cc:407] Connection header received
0618 22:31:37.748552 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:37.748561 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:37.748876 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:37.749112 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:37.750264 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.752017 (+ 1753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.754637 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.756132 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.756337 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:37.756540 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:37.756909 (+ 369us) server_negotiation.cc:299] Negotiation successful
0618 22:31:37.757106 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10707,"thread_start_us":10536,"threads_started":1}
I20260618 22:31:37.758204 28816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:37.736003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37218 (local address 127.10.242.193:35775)
0618 22:31:37.748558 (+ 12555us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:37.748567 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:37.748595 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:31:37.748694 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:37.748701 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:37.749009 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:37.750151 (+ 1142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:37.750425 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.752565 (+ 2140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.754928 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.756395 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.756601 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:37.756774 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:37.757710 (+ 936us) server_negotiation.cc:299] Negotiation successful
0618 22:31:37.757894 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12383,"thread_start_us":12243,"threads_started":1}
I20260618 22:31:37.757589 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:37.735319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:35775 (local address 127.0.0.1:37218)
0618 22:31:37.748122 (+ 12803us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:37.748140 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:37.748258 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:37.749331 (+ 1073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:37.749341 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:37.749380 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:37.749899 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:37.749914 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.752732 (+ 2818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.752737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:37.754741 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:37.754757 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.756705 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.756711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:37.756881 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:37.756914 (+ 33us) client_negotiation.cc:769] Sending connection context
0618 22:31:37.756999 (+ 85us) client_negotiation.cc:240] Negotiation successful
0618 22:31:37.757178 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12684,"mutex_wait_us":1,"thread_start_us":12529,"threads_started":1}
I20260618 22:31:37.760399 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:37.748641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:40957 (local address 127.0.0.1:60348)
0618 22:31:37.752684 (+ 4043us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:37.752703 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:37.752825 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:37.753818 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:37.753824 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:37.753852 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:37.754271 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:37.754286 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.755715 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.755719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:37.757776 (+ 2057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:37.757792 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.759603 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.759609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:37.759776 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:37.759811 (+ 35us) client_negotiation.cc:769] Sending connection context
0618 22:31:37.759889 (+ 78us) client_negotiation.cc:240] Negotiation successful
0618 22:31:37.760020 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3878,"mutex_wait_us":4,"thread_start_us":4280,"threads_started":1}
I20260618 22:31:37.761503 28817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:37.748091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60348 (local address 127.10.242.194:40957)
0618 22:31:37.751913 (+ 3822us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:37.751922 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:37.753163 (+ 1241us) server_negotiation.cc:407] Connection header received
0618 22:31:37.753280 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:37.753289 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:37.753589 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:37.753813 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:37.754463 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.755583 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.757968 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.759399 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.760736 (+ 1337us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:37.760922 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:37.761008 (+ 86us) server_negotiation.cc:299] Negotiation successful
0618 22:31:37.761195 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3660,"thread_start_us":3537,"threads_started":1}
I20260618 22:31:37.762054 28469 tablet_service.cc:1467] Processing CreateTablet for tablet b59447542b3f4aeaa9c03e576cdf1741 (DEFAULT_TABLE table=test-table [id=9c0b12744b674959be7163f289efe53b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:31:37.762074 28736 tablet_service.cc:1467] Processing CreateTablet for tablet b59447542b3f4aeaa9c03e576cdf1741 (DEFAULT_TABLE table=test-table [id=9c0b12744b674959be7163f289efe53b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:31:37.763449 28736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b59447542b3f4aeaa9c03e576cdf1741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:37.763549 28469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b59447542b3f4aeaa9c03e576cdf1741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:37.765275 28603 tablet_service.cc:1467] Processing CreateTablet for tablet b59447542b3f4aeaa9c03e576cdf1741 (DEFAULT_TABLE table=test-table [id=9c0b12744b674959be7163f289efe53b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:31:37.767179 28603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b59447542b3f4aeaa9c03e576cdf1741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:37.776269 28819 tablet_bootstrap.cc:492] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794: Bootstrap starting.
I20260618 22:31:37.778015 28820 tablet_bootstrap.cc:492] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38: Bootstrap starting.
I20260618 22:31:37.780045 28819 tablet_bootstrap.cc:654] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:37.781179 28821 tablet_bootstrap.cc:492] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29: Bootstrap starting.
I20260618 22:31:37.781412 28819 log.cc:824] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:37.781682 28820 tablet_bootstrap.cc:654] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:37.782950 28820 log.cc:824] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:37.785195 28821 tablet_bootstrap.cc:654] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:37.785202 28819 tablet_bootstrap.cc:492] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794: No bootstrap required, opened a new log
I20260618 22:31:37.785494 28819 ts_tablet_manager.cc:1397] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260618 22:31:37.786064 28820 tablet_bootstrap.cc:492] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38: No bootstrap required, opened a new log
I20260618 22:31:37.786327 28821 log.cc:824] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:37.786443 28820 ts_tablet_manager.cc:1397] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260618 22:31:37.791507 28821 tablet_bootstrap.cc:492] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29: No bootstrap required, opened a new log
I20260618 22:31:37.791793 28821 ts_tablet_manager.cc:1397] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260618 22:31:37.797372 28820 raft_consensus.cc:348] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730f147613b94fab881b09c2eb640d38" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35775 } } peers { permanent_uuid: "8ab3bf4bd234458eb721ca5f93a54794" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 33399 } } peers { permanent_uuid: "942e5af2098d4453a190a0a97834ae29" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 40957 } }
I20260618 22:31:37.797360 28819 raft_consensus.cc:348] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730f147613b94fab881b09c2eb640d38" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35775 } } peers { permanent_uuid: "8ab3bf4bd234458eb721ca5f93a54794" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 33399 } } peers { permanent_uuid: "942e5af2098d4453a190a0a97834ae29" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 40957 } }
I20260618 22:31:37.797884 28820 raft_consensus.cc:374] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:37.797895 28819 raft_consensus.cc:374] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:37.798045 28820 raft_consensus.cc:729] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730f147613b94fab881b09c2eb640d38, State: Initialized, Role: FOLLOWER
I20260618 22:31:37.797997 28819 raft_consensus.cc:729] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ab3bf4bd234458eb721ca5f93a54794, State: Initialized, Role: FOLLOWER
I20260618 22:31:37.798751 28820 consensus_queue.cc:260] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730f147613b94fab881b09c2eb640d38" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35775 } } peers { permanent_uuid: "8ab3bf4bd234458eb721ca5f93a54794" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 33399 } } peers { permanent_uuid: "942e5af2098d4453a190a0a97834ae29" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 40957 } }
I20260618 22:31:37.798754 28819 consensus_queue.cc:260] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730f147613b94fab881b09c2eb640d38" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35775 } } peers { permanent_uuid: "8ab3bf4bd234458eb721ca5f93a54794" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 33399 } } peers { permanent_uuid: "942e5af2098d4453a190a0a97834ae29" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 40957 } }
I20260618 22:31:37.800463 28821 raft_consensus.cc:348] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730f147613b94fab881b09c2eb640d38" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35775 } } peers { permanent_uuid: "8ab3bf4bd234458eb721ca5f93a54794" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 33399 } } peers { permanent_uuid: "942e5af2098d4453a190a0a97834ae29" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 40957 } }
I20260618 22:31:37.800889 28821 raft_consensus.cc:374] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:37.800998 28821 raft_consensus.cc:729] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 942e5af2098d4453a190a0a97834ae29, State: Initialized, Role: FOLLOWER
I20260618 22:31:37.801620 28821 consensus_queue.cc:260] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730f147613b94fab881b09c2eb640d38" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35775 } } peers { permanent_uuid: "8ab3bf4bd234458eb721ca5f93a54794" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 33399 } } peers { permanent_uuid: "942e5af2098d4453a190a0a97834ae29" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 40957 } }
I20260618 22:31:37.802713 28668 heartbeater.cc:503] Master 127.10.242.254:39257 was elected leader, sending a full tablet report...
I20260618 22:31:37.803496 28821 ts_tablet_manager.cc:1428] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260618 22:31:37.805111 28801 heartbeater.cc:503] Master 127.10.242.254:39257 was elected leader, sending a full tablet report...
I20260618 22:31:37.805893 28820 ts_tablet_manager.cc:1428] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38: Time spent starting tablet: real 0.019s user 0.010s sys 0.009s
I20260618 22:31:37.807097 28819 ts_tablet_manager.cc:1428] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794: Time spent starting tablet: real 0.021s user 0.018s sys 0.000s
I20260618 22:31:37.848254 28827 raft_consensus.cc:482] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:31:37.848572 28827 raft_consensus.cc:504] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730f147613b94fab881b09c2eb640d38" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35775 } } peers { permanent_uuid: "8ab3bf4bd234458eb721ca5f93a54794" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 33399 } } peers { permanent_uuid: "942e5af2098d4453a190a0a97834ae29" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 40957 } }
I20260618 22:31:37.850626 28827 leader_election.cc:288] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730f147613b94fab881b09c2eb640d38 (127.10.242.193:35775), 8ab3bf4bd234458eb721ca5f93a54794 (127.10.242.195:33399)
I20260618 22:31:37.860419 28816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:37.851119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:58643 (local address 127.10.242.193:35775)
0618 22:31:37.851330 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:37.851336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:37.852297 (+ 961us) server_negotiation.cc:407] Connection header received
0618 22:31:37.852540 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:37.852546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:37.852627 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:37.852739 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:37.853566 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.854712 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.857862 (+ 3150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.859162 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.859354 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:37.859436 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:37.860047 (+ 611us) server_negotiation.cc:299] Negotiation successful
0618 22:31:37.860185 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260618 22:31:37.860517 28828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:37.851235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:33399 (local address 127.10.242.194:40569)
0618 22:31:37.852124 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:37.852149 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:37.852349 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:37.852793 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:37.852799 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:37.852840 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:37.853384 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:37.853393 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.854932 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.854939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:37.857611 (+ 2672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:37.857635 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.859500 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.859505 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:37.859739 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:37.859817 (+ 78us) client_negotiation.cc:769] Sending connection context
0618 22:31:37.859931 (+ 114us) client_negotiation.cc:240] Negotiation successful
0618 22:31:37.860084 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"mutex_wait_us":73,"spinlock_wait_cycles":31744,"thread_start_us":441,"threads_started":1}
I20260618 22:31:37.860549 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:37.851282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:40569 (local address 127.10.242.195:33399)
0618 22:31:37.851443 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:37.851447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:37.852284 (+ 837us) server_negotiation.cc:407] Connection header received
0618 22:31:37.852540 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:37.852546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:37.852627 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:37.852739 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:37.853564 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.854720 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.857862 (+ 3142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.859168 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.859354 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:37.859462 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:37.860152 (+ 690us) server_negotiation.cc:299] Negotiation successful
0618 22:31:37.860306 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:31:37.861508 28829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:37.851235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:35775 (local address 127.10.242.194:58643)
0618 22:31:37.852194 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:37.852207 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:37.852349 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:37.852793 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:37.852799 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:37.852840 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:37.853370 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:37.853386 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.854939 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.854943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:37.857610 (+ 2667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:37.857635 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.859495 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.859504 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:37.859759 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:37.859824 (+ 65us) client_negotiation.cc:769] Sending connection context
0618 22:31:37.860937 (+ 1113us) client_negotiation.cc:240] Negotiation successful
0618 22:31:37.861119 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"mutex_wait_us":238,"spinlock_wait_cycles":18304,"thread_start_us":380,"threads_started":1}
I20260618 22:31:37.861660 28756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59447542b3f4aeaa9c03e576cdf1741" candidate_uuid: "942e5af2098d4453a190a0a97834ae29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ab3bf4bd234458eb721ca5f93a54794" is_pre_election: true
I20260618 22:31:37.862275 28756 raft_consensus.cc:2454] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 942e5af2098d4453a190a0a97834ae29 in term 0.
I20260618 22:31:37.862478 28489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59447542b3f4aeaa9c03e576cdf1741" candidate_uuid: "942e5af2098d4453a190a0a97834ae29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730f147613b94fab881b09c2eb640d38" is_pre_election: true
I20260618 22:31:37.862942 28556 leader_election.cc:302] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ab3bf4bd234458eb721ca5f93a54794, 942e5af2098d4453a190a0a97834ae29; no voters:
I20260618 22:31:37.862977 28489 raft_consensus.cc:2454] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 942e5af2098d4453a190a0a97834ae29 in term 0.
I20260618 22:31:37.863330 28827 raft_consensus.cc:2781] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:31:37.863504 28827 raft_consensus.cc:482] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:31:37.863622 28827 raft_consensus.cc:3037] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:37.865883 28827 raft_consensus.cc:504] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730f147613b94fab881b09c2eb640d38" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35775 } } peers { permanent_uuid: "8ab3bf4bd234458eb721ca5f93a54794" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 33399 } } peers { permanent_uuid: "942e5af2098d4453a190a0a97834ae29" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 40957 } }
I20260618 22:31:37.866508 28827 leader_election.cc:288] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [CANDIDATE]: Term 1 election: Requested vote from peers 730f147613b94fab881b09c2eb640d38 (127.10.242.193:35775), 8ab3bf4bd234458eb721ca5f93a54794 (127.10.242.195:33399)
I20260618 22:31:37.866992 28489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59447542b3f4aeaa9c03e576cdf1741" candidate_uuid: "942e5af2098d4453a190a0a97834ae29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730f147613b94fab881b09c2eb640d38"
I20260618 22:31:37.867038 28756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59447542b3f4aeaa9c03e576cdf1741" candidate_uuid: "942e5af2098d4453a190a0a97834ae29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ab3bf4bd234458eb721ca5f93a54794"
I20260618 22:31:37.867193 28489 raft_consensus.cc:3037] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:37.867225 28756 raft_consensus.cc:3037] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:37.869464 28756 raft_consensus.cc:2454] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 942e5af2098d4453a190a0a97834ae29 in term 1.
I20260618 22:31:37.869467 28489 raft_consensus.cc:2454] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 942e5af2098d4453a190a0a97834ae29 in term 1.
I20260618 22:31:37.869948 28558 leader_election.cc:302] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730f147613b94fab881b09c2eb640d38, 942e5af2098d4453a190a0a97834ae29; no voters:
I20260618 22:31:37.870281 28827 raft_consensus.cc:2781] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:31:37.870719 28827 raft_consensus.cc:686] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [term 1 LEADER]: Becoming Leader. State: Replica: 942e5af2098d4453a190a0a97834ae29, State: Running, Role: LEADER
I20260618 22:31:37.871225 28827 consensus_queue.cc:237] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730f147613b94fab881b09c2eb640d38" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35775 } } peers { permanent_uuid: "8ab3bf4bd234458eb721ca5f93a54794" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 33399 } } peers { permanent_uuid: "942e5af2098d4453a190a0a97834ae29" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 40957 } }
I20260618 22:31:37.875814 28345 catalog_manager.cc:5495] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 reported cstate change: term changed from 0 to 1, leader changed from <none> to 942e5af2098d4453a190a0a97834ae29 (127.10.242.194). New cstate: current_term: 1 leader_uuid: "942e5af2098d4453a190a0a97834ae29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730f147613b94fab881b09c2eb640d38" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ab3bf4bd234458eb721ca5f93a54794" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 33399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "942e5af2098d4453a190a0a97834ae29" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260618 22:31:37.905017 28817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:37.899480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60350 (local address 127.10.242.194:40957)
0618 22:31:37.899719 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:37.899724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:37.899750 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:31:37.900029 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:37.900036 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:37.900131 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:37.900262 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:31:37.900648 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.901474 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.902892 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.903284 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.903402 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:37.903878 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:31:37.904445 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:31:37.904581 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:37.904666 (+ 85us) server_negotiation.cc:299] Negotiation successful
0618 22:31:37.904748 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260618 22:31:37.909430 28802 tablet.cc:2369] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:31:37.913578 28489 raft_consensus.cc:1261] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38 [term 1 FOLLOWER]: Refusing update from remote peer 942e5af2098d4453a190a0a97834ae29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:31:37.913611 28756 raft_consensus.cc:1261] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794 [term 1 FOLLOWER]: Refusing update from remote peer 942e5af2098d4453a190a0a97834ae29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260618 22:31:37.914402 28535 tablet.cc:2369] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:31:37.914446 28669 tablet.cc:2369] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:31:37.914564 28827 consensus_queue.cc:1035] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ab3bf4bd234458eb721ca5f93a54794" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 33399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:31:37.914870 28830 consensus_queue.cc:1035] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730f147613b94fab881b09c2eb640d38" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:31:37.941612 28834 mvcc.cc:205] Tried to move back new op lower bound from 7298342493843623936 to 7298342493684547584. Current Snapshot: MvccSnapshot[applied={T|T < 7298342493843623936}]
I20260618 22:31:37.943163 28345 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:31:37.943480 28345 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:31:37.947342 28345 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1bfc9f90692b4babb1956ffbab6204a3: Sending DeleteTablet for 3 replicas of tablet b59447542b3f4aeaa9c03e576cdf1741
I20260618 22:31:37.948709 28736 tablet_service.cc:1514] Processing DeleteTablet for tablet b59447542b3f4aeaa9c03e576cdf1741 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:31:37 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:57450
I20260618 22:31:37.948845 28469 tablet_service.cc:1514] Processing DeleteTablet for tablet b59447542b3f4aeaa9c03e576cdf1741 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:31:37 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:37218
I20260618 22:31:37.948916 28603 tablet_service.cc:1514] Processing DeleteTablet for tablet b59447542b3f4aeaa9c03e576cdf1741 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:31:37 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:60348
I20260618 22:31:37.950738 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 28406
I20260618 22:31:37.962883 28843 tablet_replica.cc:330] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38: stopping tablet replica
I20260618 22:31:37.963451 28843 raft_consensus.cc:2229] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:31:37.964061 28843 raft_consensus.cc:2258] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:31:37.966598 28844 tablet_replica.cc:330] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29: stopping tablet replica
I20260618 22:31:37.967073 28844 raft_consensus.cc:2229] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:31:37.967792 28844 raft_consensus.cc:2258] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:31:37.968441 28843 ts_tablet_manager.cc:1905] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:31:37.970911 28842 tablet_replica.cc:330] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794: stopping tablet replica
I20260618 22:31:37.971364 28842 raft_consensus.cc:2229] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:31:37.971899 28842 raft_consensus.cc:2258] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:31:37.972313 28844 ts_tablet_manager.cc:1905] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 18 22:31:37 dist-test-slave-0956 krb5kdc[28300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:31:37.978559 28843 ts_tablet_manager.cc:1918] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:31:37.978731 28843 log.cc:1196] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/b59447542b3f4aeaa9c03e576cdf1741
I20260618 22:31:37.979008 28842 ts_tablet_manager.cc:1905] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:31:37.979254 28843 ts_tablet_manager.cc:1939] T b59447542b3f4aeaa9c03e576cdf1741 P 730f147613b94fab881b09c2eb640d38: Deleting consensus metadata
I20260618 22:31:37.981166 28331 catalog_manager.cc:4841] TS 730f147613b94fab881b09c2eb640d38 (127.10.242.193:35775): tablet b59447542b3f4aeaa9c03e576cdf1741 (table test-table [id=9c0b12744b674959be7163f289efe53b]) successfully deleted
I20260618 22:31:37.981741 28844 ts_tablet_manager.cc:1918] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:31:37.981913 28844 log.cc:1196] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/b59447542b3f4aeaa9c03e576cdf1741
I20260618 22:31:37.982535 28844 ts_tablet_manager.cc:1939] T b59447542b3f4aeaa9c03e576cdf1741 P 942e5af2098d4453a190a0a97834ae29: Deleting consensus metadata
I20260618 22:31:37.982856 28816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:37.952205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37226 (local address 127.10.242.193:35775)
0618 22:31:37.952417 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:37.952424 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:37.957608 (+ 5184us) server_negotiation.cc:407] Connection header received
0618 22:31:37.958627 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:37.958635 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:37.958729 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:37.958804 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:37.964757 (+ 5953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.965949 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.968409 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:37.969158 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:37.969271 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:37.975348 (+ 6077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:37.975367 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:37.975386 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:37.975460 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:37.979397 (+ 3937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:37.980368 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:37.980378 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:37.980381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:37.980563 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:37.981013 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:37.981018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:37.981021 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:37.981511 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:37.981775 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:37.982247 (+ 472us) server_negotiation.cc:299] Negotiation successful
0618 22:31:37.982484 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260618 22:31:37.984364 28529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:37.984339 28329 catalog_manager.cc:4841] TS 942e5af2098d4453a190a0a97834ae29 (127.10.242.194:40957): tablet b59447542b3f4aeaa9c03e576cdf1741 (table test-table [id=9c0b12744b674959be7163f289efe53b]) successfully deleted
I20260618 22:31:37.986516 28842 ts_tablet_manager.cc:1918] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:31:37.986747 28842 log.cc:1196] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/b59447542b3f4aeaa9c03e576cdf1741
I20260618 22:31:37.987367 28842 ts_tablet_manager.cc:1939] T b59447542b3f4aeaa9c03e576cdf1741 P 8ab3bf4bd234458eb721ca5f93a54794: Deleting consensus metadata
I20260618 22:31:37.989521 28329 catalog_manager.cc:4841] TS 8ab3bf4bd234458eb721ca5f93a54794 (127.10.242.195:33399): tablet b59447542b3f4aeaa9c03e576cdf1741 (table test-table [id=9c0b12744b674959be7163f289efe53b]) successfully deleted
I20260618 22:31:38.073719 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 28406
I20260618 22:31:38.088855 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 28540
Jun 18 22:31:38 dist-test-slave-0956 krb5kdc[28300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.194@KRBTEST.COM
I20260618 22:31:38.107023 28817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:38.089640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60358 (local address 127.10.242.194:40957)
0618 22:31:38.089806 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:38.089811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:38.089848 (+ 37us) server_negotiation.cc:407] Connection header received
0618 22:31:38.090083 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:38.090087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:38.090171 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:38.090251 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:38.091893 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:38.092745 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:38.094771 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:38.095249 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:38.095351 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:38.100684 (+ 5333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:38.100700 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:38.100715 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:38.100768 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:38.103826 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:38.104601 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:38.104610 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:38.104612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:38.104770 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:38.105422 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:38.105428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:38.105430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:38.105814 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:38.106123 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:38.106445 (+ 322us) server_negotiation.cc:299] Negotiation successful
0618 22:31:38.106675 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260618 22:31:38.107658 28663 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:38.217986 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 28540
I20260618 22:31:38.232990 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 28673
Jun 18 22:31:38 dist-test-slave-0956 krb5kdc[28300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.195@KRBTEST.COM
I20260618 22:31:38.247865 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:38.233732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57462 (local address 127.10.242.195:33399)
0618 22:31:38.233965 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:38.233971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:38.233990 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:31:38.234143 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:38.234149 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:38.234224 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:38.234306 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:38.235605 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:38.236329 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:38.237612 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:38.238025 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:38.238132 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:38.242417 (+ 4285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:38.242433 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:38.242447 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:38.242481 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:38.245196 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:38.245885 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:38.245893 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:38.245895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:38.246039 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:38.246391 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:38.246394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:38.246396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:38.246753 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:38.247026 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:38.247332 (+ 306us) server_negotiation.cc:299] Negotiation successful
0618 22:31:38.247548 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260618 22:31:38.248514 28796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:38.336164 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 28673
I20260618 22:31:38.349107 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 28313
I20260618 22:31:38.350184 28375 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:38.459447 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 28313
2026-06-18T22:31:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4088 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-18T22:31:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:31:38Z Disabled control of system clock
I20260618 22:31:38.529109 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:45687
--webserver_interface=127.10.242.254
--webserver_port=0
--builtin_ntp_servers=127.10.242.212:34337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:45687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:31:38.855024 28853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:38.855325 28853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:38.855417 28853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:38.862880 28853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:31:38.862998 28853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:38.863035 28853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:38.863090 28853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:31:38.863137 28853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:31:38.872473 28853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:45687
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:45687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:38.873786 28853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:38.875599 28853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:38.887183 28861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:38.887190 28859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:38.887542 28858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:38.888046 28853 server_base.cc:1034] running on GCE node
I20260618 22:31:38.888794 28853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:38.890276 28853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:38.891484 28853 hybrid_clock.cc:648] HybridClock initialized: now 1781821898891443 us; error 36 us; skew 500 ppm
I20260618 22:31:38.891829 28853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:31:38.894107 28853 webserver.cc:460] Webserver started at http://127.10.242.254:33037/ using document root <none> and password file <none>
I20260618 22:31:38.894768 28853 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:38.894867 28853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:38.895179 28853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:38.898161 28853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "647a435ebda94778b7ee7d499083b887"
format_stamp: "Formatted at 2026-06-18 22:31:38 on dist-test-slave-0956"
server_key: "d72c69b449ef9c9fa5f18dc753132bca"
server_key_iv: "4deac76654fe6e3d630b0d7f3a9037d2"
server_key_version: "encryptionkey@0"
I20260618 22:31:38.898979 28853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "647a435ebda94778b7ee7d499083b887"
format_stamp: "Formatted at 2026-06-18 22:31:38 on dist-test-slave-0956"
server_key: "d72c69b449ef9c9fa5f18dc753132bca"
server_key_iv: "4deac76654fe6e3d630b0d7f3a9037d2"
server_key_version: "encryptionkey@0"
I20260618 22:31:38.904940 28853 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260618 22:31:38.908458 28867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:38.909929 28853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:31:38.910127 28853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "647a435ebda94778b7ee7d499083b887"
format_stamp: "Formatted at 2026-06-18 22:31:38 on dist-test-slave-0956"
server_key: "d72c69b449ef9c9fa5f18dc753132bca"
server_key_iv: "4deac76654fe6e3d630b0d7f3a9037d2"
server_key_version: "encryptionkey@0"
I20260618 22:31:38.910328 28853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:38.931478 28853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:38.932374 28853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:38.932659 28853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:38.950121 28853 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:45687
I20260618 22:31:38.950139 28918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:45687 every 8 connection(s)
I20260618 22:31:38.951469 28853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:31:38.952005 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 28853
I20260618 22:31:38.952173 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:31:38.952744 11211 external_mini_cluster.cc:1428] Setting key fd06439e63c5b6b58fdba7ed793901e0
I20260618 22:31:38.955685 28919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:38.966653 28922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:38.954766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41076 (local address 127.10.242.254:45687)
0618 22:31:38.955902 (+ 1136us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:38.955913 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:38.955965 (+ 52us) server_negotiation.cc:407] Connection header received
0618 22:31:38.957620 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:38.957641 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:38.958102 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:38.958450 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:38.959042 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:38.960352 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:38.962319 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:38.963024 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:38.963162 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:38.964335 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:38.964342 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:38.964359 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:38.964913 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:38.965003 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:38.965024 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:38.965157 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:38.965350 (+ 193us) server_negotiation.cc:299] Negotiation successful
0618 22:31:38.965685 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":477,"threads_started":1}
I20260618 22:31:38.967545 28919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887: Bootstrap starting.
I20260618 22:31:38.970993 28919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:38.971952 28919 log.cc:824] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:38.974757 28919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887: No bootstrap required, opened a new log
I20260618 22:31:38.982590 28919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "647a435ebda94778b7ee7d499083b887" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45687 } }
I20260618 22:31:38.982990 28919 raft_consensus.cc:374] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:38.983086 28919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a435ebda94778b7ee7d499083b887, State: Initialized, Role: FOLLOWER
I20260618 22:31:38.983624 28919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "647a435ebda94778b7ee7d499083b887" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45687 } }
I20260618 22:31:38.983832 28919 raft_consensus.cc:388] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:31:38.983963 28919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:31:38.984107 28919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:38.986372 28919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "647a435ebda94778b7ee7d499083b887" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45687 } }
I20260618 22:31:38.986853 28919 leader_election.cc:302] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 647a435ebda94778b7ee7d499083b887; no voters:
I20260618 22:31:38.987468 28919 leader_election.cc:288] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:31:38.987617 28924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:31:38.988390 28924 raft_consensus.cc:686] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887 [term 1 LEADER]: Becoming Leader. State: Replica: 647a435ebda94778b7ee7d499083b887, State: Running, Role: LEADER
I20260618 22:31:38.988862 28924 consensus_queue.cc:237] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "647a435ebda94778b7ee7d499083b887" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45687 } }
I20260618 22:31:38.989432 28919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:31:38.991231 28926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 647a435ebda94778b7ee7d499083b887. Latest consensus state: current_term: 1 leader_uuid: "647a435ebda94778b7ee7d499083b887" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "647a435ebda94778b7ee7d499083b887" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45687 } } }
I20260618 22:31:38.991726 28926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887 [sys.catalog]: This master's current role is: LEADER
I20260618 22:31:38.991578 28925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "647a435ebda94778b7ee7d499083b887" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "647a435ebda94778b7ee7d499083b887" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45687 } } }
I20260618 22:31:38.993248 28925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887 [sys.catalog]: This master's current role is: LEADER
I20260618 22:31:38.995293 28931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:31:39.000785 28931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:31:39.009119 28931 catalog_manager.cc:1347] Generated new cluster ID: a94419390eae454da10588608691807f
I20260618 22:31:39.009310 28931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:31:39.025599 28931 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:31:39.027454 28931 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:31:39.034209 28931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887: Generated new TSK 0
I20260618 22:31:39.035303 28931 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:31:39.046468 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45687
--builtin_ntp_servers=127.10.242.212:34337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:31:39.371483 28943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:39.371773 28943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:39.371887 28943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:39.381392 28943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:39.381536 28943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:39.381700 28943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:31:39.392335 28943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:39.393721 28943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:39.395509 28943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:39.408205 28949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:39.408360 28948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:39.408243 28951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:39.408928 28943 server_base.cc:1034] running on GCE node
I20260618 22:31:39.409588 28943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:39.410547 28943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:39.411746 28943 hybrid_clock.cc:648] HybridClock initialized: now 1781821899411699 us; error 38 us; skew 500 ppm
I20260618 22:31:39.412075 28943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:31:39.414603 28943 webserver.cc:460] Webserver started at http://127.10.242.193:44017/ using document root <none> and password file <none>
I20260618 22:31:39.415349 28943 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:39.415441 28943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:39.415747 28943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:39.418680 28943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "222eb63b4bd2460bbb8337d7efd5449e"
format_stamp: "Formatted at 2026-06-18 22:31:39 on dist-test-slave-0956"
server_key: "c782874d1c63d860199b78db309fa681"
server_key_iv: "d92cbf24e359050e97849b5523410f21"
server_key_version: "encryptionkey@0"
I20260618 22:31:39.419523 28943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "222eb63b4bd2460bbb8337d7efd5449e"
format_stamp: "Formatted at 2026-06-18 22:31:39 on dist-test-slave-0956"
server_key: "c782874d1c63d860199b78db309fa681"
server_key_iv: "d92cbf24e359050e97849b5523410f21"
server_key_version: "encryptionkey@0"
I20260618 22:31:39.424889 28943 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260618 22:31:39.428517 28957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:39.430210 28943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:31:39.430379 28943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "222eb63b4bd2460bbb8337d7efd5449e"
format_stamp: "Formatted at 2026-06-18 22:31:39 on dist-test-slave-0956"
server_key: "c782874d1c63d860199b78db309fa681"
server_key_iv: "d92cbf24e359050e97849b5523410f21"
server_key_version: "encryptionkey@0"
I20260618 22:31:39.430583 28943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:39.453871 28943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:39.454716 28943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:39.455006 28943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:39.455974 28943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:39.457464 28943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:39.457548 28943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:39.457620 28943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:39.457687 28943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:39.487550 28943 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:36923
I20260618 22:31:39.487620 29069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:36923 every 8 connection(s)
I20260618 22:31:39.488828 28943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:31:39.494189 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 28943
I20260618 22:31:39.494367 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:31:39.494946 11211 external_mini_cluster.cc:1428] Setting key eda8ad673649f24a33b152f11ab58cab
I20260618 22:31:39.497962 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45687
--builtin_ntp_servers=127.10.242.212:34337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260618 22:31:39.502712 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:39.491516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:39231 (local address 127.10.242.254:45687)
0618 22:31:39.491973 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:39.491977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:39.493115 (+ 1138us) server_negotiation.cc:407] Connection header received
0618 22:31:39.495363 (+ 2248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:39.495368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:39.495444 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:39.495557 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:39.496702 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:39.497702 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:39.499509 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:39.500065 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:39.500183 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:39.501791 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:39.501797 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:39.501802 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:39.501826 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:39.501885 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:39.501900 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:39.501989 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:39.502257 (+ 268us) server_negotiation.cc:299] Negotiation successful
0618 22:31:39.502360 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":216,"threads_started":1}
I20260618 22:31:39.503695 29073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:39.492025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45687 (local address 127.10.242.193:39231)
0618 22:31:39.492940 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:39.492984 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:39.495056 (+ 2072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:39.495807 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:39.495818 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:39.495949 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:31:39.496521 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:39.496540 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:39.497895 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:39.497900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:39.499360 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:39.499378 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:39.500308 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:39.500318 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:39.500556 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:39.501391 (+ 835us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:31:39.501394 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:31:39.501439 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:31:39.501558 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:31:39.502030 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:31:39.502041 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:31:39.502048 (+ 7us) client_negotiation.cc:769] Sending connection context
0618 22:31:39.502226 (+ 178us) client_negotiation.cc:240] Negotiation successful
0618 22:31:39.502586 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":301,"threads_started":1}
I20260618 22:31:39.505455 29070 heartbeater.cc:347] Connected to a master server at 127.10.242.254:45687
I20260618 22:31:39.505749 29070 heartbeater.cc:464] Registering TS with master...
I20260618 22:31:39.506531 29070 heartbeater.cc:511] Master 127.10.242.254:45687 requested a full tablet report, sending...
I20260618 22:31:39.508659 28884 ts_manager.cc:194] Registered new tserver with Master: 222eb63b4bd2460bbb8337d7efd5449e (127.10.242.193:36923)
I20260618 22:31:39.510172 28884 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.193:39231
W20260618 22:31:39.821528 29074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:39.821839 29074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:39.821979 29074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:39.829836 29074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:39.829975 29074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:39.830116 29074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:31:39.839699 29074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:39.841017 29074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:39.842931 29074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:39.856716 29079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:39.856885 29082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:39.857115 29080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:39.858152 29074 server_base.cc:1034] running on GCE node
I20260618 22:31:39.858814 29074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:39.859840 29074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:39.861125 29074 hybrid_clock.cc:648] HybridClock initialized: now 1781821899861032 us; error 58 us; skew 500 ppm
I20260618 22:31:39.861481 29074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:31:39.864280 29074 webserver.cc:460] Webserver started at http://127.10.242.194:43241/ using document root <none> and password file <none>
I20260618 22:31:39.865139 29074 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:39.865255 29074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:39.865618 29074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:39.868826 29074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "4497eb46020e4844ba1c66e28896c3b1"
format_stamp: "Formatted at 2026-06-18 22:31:39 on dist-test-slave-0956"
server_key: "a9cacbbaa3817aff26fa0c550db33318"
server_key_iv: "1d87539e0df3a022e2fe98560403dd54"
server_key_version: "encryptionkey@0"
I20260618 22:31:39.869745 29074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "4497eb46020e4844ba1c66e28896c3b1"
format_stamp: "Formatted at 2026-06-18 22:31:39 on dist-test-slave-0956"
server_key: "a9cacbbaa3817aff26fa0c550db33318"
server_key_iv: "1d87539e0df3a022e2fe98560403dd54"
server_key_version: "encryptionkey@0"
I20260618 22:31:39.875335 29074 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:31:39.879251 29088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:39.880760 29074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260618 22:31:39.880990 29074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "4497eb46020e4844ba1c66e28896c3b1"
format_stamp: "Formatted at 2026-06-18 22:31:39 on dist-test-slave-0956"
server_key: "a9cacbbaa3817aff26fa0c550db33318"
server_key_iv: "1d87539e0df3a022e2fe98560403dd54"
server_key_version: "encryptionkey@0"
I20260618 22:31:39.881283 29074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:39.897631 29074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:39.898486 29074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:39.898777 29074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:39.899781 29074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:39.901382 29074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:39.901500 29074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:39.901593 29074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:39.901647 29074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:39.931250 29074 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:33885
I20260618 22:31:39.931294 29200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:33885 every 8 connection(s)
I20260618 22:31:39.932542 29074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:31:39.942375 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 29074
I20260618 22:31:39.942528 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:31:39.942942 11211 external_mini_cluster.cc:1428] Setting key 83e0e19089ab50d50cd0267f27991932
I20260618 22:31:39.945255 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:39.935061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:56443 (local address 127.10.242.254:45687)
0618 22:31:39.935290 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:39.935294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:39.936747 (+ 1453us) server_negotiation.cc:407] Connection header received
0618 22:31:39.938732 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:39.938737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:39.938827 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:39.938912 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:39.940070 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:39.940775 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:39.942421 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:39.942896 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:39.943011 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:39.944338 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:39.944343 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:39.944347 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:39.944366 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:39.944397 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:39.944417 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:39.944503 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:39.944813 (+ 310us) server_negotiation.cc:299] Negotiation successful
0618 22:31:39.944916 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260618 22:31:39.945879 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45687
--builtin_ntp_servers=127.10.242.212:34337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260618 22:31:39.946310 29203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:39.935596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45687 (local address 127.10.242.194:56443)
0618 22:31:39.936540 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:39.936614 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:39.938442 (+ 1828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:39.939190 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:39.939200 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:39.939301 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:31:39.939886 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:39.939903 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:39.940941 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:39.940944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:39.942265 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:39.942279 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:39.943122 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:39.943130 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:39.943368 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:39.944022 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:31:39.944024 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:31:39.944067 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:31:39.944150 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:31:39.944553 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:31:39.944565 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:31:39.944574 (+ 9us) client_negotiation.cc:769] Sending connection context
0618 22:31:39.944783 (+ 209us) client_negotiation.cc:240] Negotiation successful
0618 22:31:39.945181 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":292,"threads_started":1}
I20260618 22:31:39.947705 29201 heartbeater.cc:347] Connected to a master server at 127.10.242.254:45687
I20260618 22:31:39.947979 29201 heartbeater.cc:464] Registering TS with master...
I20260618 22:31:39.948736 29201 heartbeater.cc:511] Master 127.10.242.254:45687 requested a full tablet report, sending...
I20260618 22:31:39.950134 28884 ts_manager.cc:194] Registered new tserver with Master: 4497eb46020e4844ba1c66e28896c3b1 (127.10.242.194:33885)
I20260618 22:31:39.950947 28884 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.194:56443
W20260618 22:31:40.270864 29204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:40.271162 29204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:40.271297 29204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:40.278650 29204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:40.278780 29204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:40.278925 29204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:31:40.288239 29204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:45687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:40.289659 29204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:40.291539 29204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:40.304378 29210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:40.304407 29209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:40.304382 29212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:40.305325 29204 server_base.cc:1034] running on GCE node
I20260618 22:31:40.305907 29204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:40.306866 29204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:40.308072 29204 hybrid_clock.cc:648] HybridClock initialized: now 1781821900308016 us; error 40 us; skew 500 ppm
I20260618 22:31:40.308406 29204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:31:40.310750 29204 webserver.cc:460] Webserver started at http://127.10.242.195:38251/ using document root <none> and password file <none>
I20260618 22:31:40.311493 29204 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:40.311589 29204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:40.311885 29204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:40.314877 29204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "d9c5c1dca7054362bc660f01ac1e25ce"
format_stamp: "Formatted at 2026-06-18 22:31:40 on dist-test-slave-0956"
server_key: "c36b7f558e8f12489ffce620b64412dd"
server_key_iv: "8c282aa1c2d270b9555d23f7ed293ca0"
server_key_version: "encryptionkey@0"
I20260618 22:31:40.315665 29204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "d9c5c1dca7054362bc660f01ac1e25ce"
format_stamp: "Formatted at 2026-06-18 22:31:40 on dist-test-slave-0956"
server_key: "c36b7f558e8f12489ffce620b64412dd"
server_key_iv: "8c282aa1c2d270b9555d23f7ed293ca0"
server_key_version: "encryptionkey@0"
I20260618 22:31:40.320845 29204 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260618 22:31:40.324362 29218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:40.325732 29204 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260618 22:31:40.325881 29204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "d9c5c1dca7054362bc660f01ac1e25ce"
format_stamp: "Formatted at 2026-06-18 22:31:40 on dist-test-slave-0956"
server_key: "c36b7f558e8f12489ffce620b64412dd"
server_key_iv: "8c282aa1c2d270b9555d23f7ed293ca0"
server_key_version: "encryptionkey@0"
I20260618 22:31:40.326081 29204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:40.344089 29204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:40.344888 29204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:40.345192 29204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:40.346160 29204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:40.347616 29204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:40.347705 29204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:40.347810 29204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:40.347883 29204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:40.377774 29204 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:43575
I20260618 22:31:40.377854 29330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:43575 every 8 connection(s)
I20260618 22:31:40.379091 29204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:31:40.382853 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 29204
I20260618 22:31:40.383037 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:31:40.383607 11211 external_mini_cluster.cc:1428] Setting key e941557fa4a53862b5d6cc0a9c6e38f7
I20260618 22:31:40.391721 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:40.381497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:60987 (local address 127.10.242.254:45687)
0618 22:31:40.381707 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:40.381713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:40.383241 (+ 1528us) server_negotiation.cc:407] Connection header received
0618 22:31:40.385513 (+ 2272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:40.385519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:40.385603 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:40.385696 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:40.386855 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.387558 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.389230 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.389747 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.389849 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:40.391115 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:40.391120 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:40.391123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:40.391140 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:40.391170 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:40.391181 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:40.391249 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:40.391409 (+ 160us) server_negotiation.cc:299] Negotiation successful
0618 22:31:40.391483 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260618 22:31:40.392695 29333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:40.382028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:45687 (local address 127.10.242.195:60987)
0618 22:31:40.383078 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:40.383126 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:40.385209 (+ 2083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:40.385878 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:40.385886 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:40.386022 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:31:40.386719 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:40.386737 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.387665 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.387669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:40.389100 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:40.389116 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.389992 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.389999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:40.390190 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:40.390790 (+ 600us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:31:40.390792 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:31:40.390832 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:31:40.390940 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:31:40.391253 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:31:40.391259 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:31:40.391264 (+ 5us) client_negotiation.cc:769] Sending connection context
0618 22:31:40.391402 (+ 138us) client_negotiation.cc:240] Negotiation successful
0618 22:31:40.391686 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":265,"threads_started":1}
I20260618 22:31:40.393961 29331 heartbeater.cc:347] Connected to a master server at 127.10.242.254:45687
I20260618 22:31:40.394241 29331 heartbeater.cc:464] Registering TS with master...
I20260618 22:31:40.394979 29331 heartbeater.cc:511] Master 127.10.242.254:45687 requested a full tablet report, sending...
I20260618 22:31:40.396291 28884 ts_manager.cc:194] Registered new tserver with Master: d9c5c1dca7054362bc660f01ac1e25ce (127.10.242.195:43575)
I20260618 22:31:40.397220 28884 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.195:60987
I20260618 22:31:40.398612 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:31:40.408208 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:40.402737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41080 (local address 127.10.242.254:45687)
0618 22:31:40.402928 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:40.402934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:40.403243 (+ 309us) server_negotiation.cc:407] Connection header received
0618 22:31:40.403406 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:40.403411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:40.403478 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:40.403559 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:40.404043 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.404715 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.406120 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.406490 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.406595 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:40.407565 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:40.407571 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:40.407574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:40.407590 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:40.407620 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:40.407631 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:40.407702 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:40.407825 (+ 123us) server_negotiation.cc:299] Negotiation successful
0618 22:31:40.407919 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260618 22:31:40.411754 28884 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:31:40.418587 28884 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:31:40.442076 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:40.430200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:36923 (local address 127.0.0.1:45554)
0618 22:31:40.431754 (+ 1554us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:40.431776 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:40.431877 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:40.432848 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:40.432856 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:40.432885 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:40.434215 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:40.434229 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.436176 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.436182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:40.439258 (+ 3076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:40.439273 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.440889 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.440897 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:40.441340 (+ 443us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:40.441379 (+ 39us) client_negotiation.cc:769] Sending connection context
0618 22:31:40.441456 (+ 77us) client_negotiation.cc:240] Negotiation successful
0618 22:31:40.441592 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1422,"mutex_wait_us":20,"thread_start_us":700,"threads_started":1}
I20260618 22:31:40.442250 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:40.430184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:43575 (local address 127.0.0.1:55980)
0618 22:31:40.430936 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:40.430986 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:40.431175 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:40.433661 (+ 2486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:40.433666 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:40.433689 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:40.434347 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:40.434360 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.436235 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.436240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:40.438992 (+ 2752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:40.439007 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.440736 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.440742 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:40.440982 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:40.441042 (+ 60us) client_negotiation.cc:769] Sending connection context
0618 22:31:40.441250 (+ 208us) client_negotiation.cc:240] Negotiation successful
0618 22:31:40.441463 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":67,"thread_start_us":1260,"threads_started":1}
I20260618 22:31:40.442523 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:40.430718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45554 (local address 127.10.242.193:36923)
0618 22:31:40.431955 (+ 1237us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:40.431964 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:40.431993 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:31:40.432086 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:40.432093 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:40.432339 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:40.432547 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:40.434442 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.436012 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.439448 (+ 3436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.440689 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.440845 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:40.441066 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:40.442015 (+ 949us) server_negotiation.cc:299] Negotiation successful
0618 22:31:40.442196 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1100,"thread_start_us":995,"threads_started":1}
I20260618 22:31:40.442538 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:40.430286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55980 (local address 127.10.242.195:43575)
0618 22:31:40.432820 (+ 2534us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:40.432828 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:40.432858 (+ 30us) server_negotiation.cc:407] Connection header received
0618 22:31:40.432944 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:40.432951 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:40.433216 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:40.433401 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:40.434530 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.436096 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.439199 (+ 3103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.440194 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.440377 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:40.440613 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:40.441975 (+ 1362us) server_negotiation.cc:299] Negotiation successful
0618 22:31:40.442164 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2372,"thread_start_us":2231,"threads_started":1}
I20260618 22:31:40.445802 29266 tablet_service.cc:1467] Processing CreateTablet for tablet 7e36ddd84dc04ff98b89391c5d5366ba (DEFAULT_TABLE table=test-table [id=5f37f315103940df830f5c30576f772c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:31:40.445801 29005 tablet_service.cc:1467] Processing CreateTablet for tablet 7e36ddd84dc04ff98b89391c5d5366ba (DEFAULT_TABLE table=test-table [id=5f37f315103940df830f5c30576f772c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:31:40.447243 29266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e36ddd84dc04ff98b89391c5d5366ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:40.447470 29005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e36ddd84dc04ff98b89391c5d5366ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:40.448694 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:40.432616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:33885 (local address 127.0.0.1:39810)
0618 22:31:40.437173 (+ 4557us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:40.437192 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:40.437316 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:40.438212 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:40.438217 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:40.438245 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:40.438640 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:40.438652 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.440324 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.440329 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:40.443054 (+ 2725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:40.443070 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.447888 (+ 4818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.447895 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:40.448075 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:40.448109 (+ 34us) client_negotiation.cc:769] Sending connection context
0618 22:31:40.448208 (+ 99us) client_negotiation.cc:240] Negotiation successful
0618 22:31:40.448357 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4425,"thread_start_us":4298,"threads_started":1}
I20260618 22:31:40.448796 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:40.433122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39810 (local address 127.10.242.194:33885)
0618 22:31:40.437173 (+ 4051us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:40.437181 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:40.437288 (+ 107us) server_negotiation.cc:407] Connection header received
0618 22:31:40.437468 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:40.437476 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:40.437730 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:40.437940 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:40.438825 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.440175 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.443241 (+ 3066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.447675 (+ 4434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.447850 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:40.448072 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:40.448280 (+ 208us) server_negotiation.cc:299] Negotiation successful
0618 22:31:40.448455 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3519,"thread_start_us":3371,"threads_started":1}
I20260618 22:31:40.452929 29136 tablet_service.cc:1467] Processing CreateTablet for tablet 7e36ddd84dc04ff98b89391c5d5366ba (DEFAULT_TABLE table=test-table [id=5f37f315103940df830f5c30576f772c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:31:40.454536 29136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e36ddd84dc04ff98b89391c5d5366ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:40.459383 29349 tablet_bootstrap.cc:492] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e: Bootstrap starting.
I20260618 22:31:40.461769 29350 tablet_bootstrap.cc:492] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce: Bootstrap starting.
I20260618 22:31:40.464366 29349 tablet_bootstrap.cc:654] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:40.465318 29350 tablet_bootstrap.cc:654] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:40.465754 29349 log.cc:824] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:40.466562 29350 log.cc:824] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:40.467952 29353 tablet_bootstrap.cc:492] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1: Bootstrap starting.
I20260618 22:31:40.469293 29349 tablet_bootstrap.cc:492] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e: No bootstrap required, opened a new log
I20260618 22:31:40.469580 29349 ts_tablet_manager.cc:1397] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260618 22:31:40.471390 29353 tablet_bootstrap.cc:654] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:40.471837 29350 tablet_bootstrap.cc:492] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce: No bootstrap required, opened a new log
I20260618 22:31:40.472112 29350 ts_tablet_manager.cc:1397] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260618 22:31:40.472421 29353 log.cc:824] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:40.474835 29353 tablet_bootstrap.cc:492] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1: No bootstrap required, opened a new log
I20260618 22:31:40.475071 29353 ts_tablet_manager.cc:1397] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260618 22:31:40.480152 29349 raft_consensus.cc:348] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c5c1dca7054362bc660f01ac1e25ce" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43575 } } peers { permanent_uuid: "222eb63b4bd2460bbb8337d7efd5449e" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36923 } } peers { permanent_uuid: "4497eb46020e4844ba1c66e28896c3b1" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33885 } }
I20260618 22:31:40.480875 29349 raft_consensus.cc:374] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:40.481040 29349 raft_consensus.cc:729] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 222eb63b4bd2460bbb8337d7efd5449e, State: Initialized, Role: FOLLOWER
I20260618 22:31:40.481973 29349 consensus_queue.cc:260] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c5c1dca7054362bc660f01ac1e25ce" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43575 } } peers { permanent_uuid: "222eb63b4bd2460bbb8337d7efd5449e" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36923 } } peers { permanent_uuid: "4497eb46020e4844ba1c66e28896c3b1" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33885 } }
I20260618 22:31:40.482391 29353 raft_consensus.cc:348] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c5c1dca7054362bc660f01ac1e25ce" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43575 } } peers { permanent_uuid: "222eb63b4bd2460bbb8337d7efd5449e" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36923 } } peers { permanent_uuid: "4497eb46020e4844ba1c66e28896c3b1" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33885 } }
I20260618 22:31:40.482422 29350 raft_consensus.cc:348] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c5c1dca7054362bc660f01ac1e25ce" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43575 } } peers { permanent_uuid: "222eb63b4bd2460bbb8337d7efd5449e" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36923 } } peers { permanent_uuid: "4497eb46020e4844ba1c66e28896c3b1" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33885 } }
I20260618 22:31:40.482926 29350 raft_consensus.cc:374] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:40.483008 29353 raft_consensus.cc:374] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:40.483067 29350 raft_consensus.cc:729] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9c5c1dca7054362bc660f01ac1e25ce, State: Initialized, Role: FOLLOWER
I20260618 22:31:40.483165 29353 raft_consensus.cc:729] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4497eb46020e4844ba1c66e28896c3b1, State: Initialized, Role: FOLLOWER
I20260618 22:31:40.483281 29070 heartbeater.cc:503] Master 127.10.242.254:45687 was elected leader, sending a full tablet report...
I20260618 22:31:40.483879 29350 consensus_queue.cc:260] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c5c1dca7054362bc660f01ac1e25ce" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43575 } } peers { permanent_uuid: "222eb63b4bd2460bbb8337d7efd5449e" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36923 } } peers { permanent_uuid: "4497eb46020e4844ba1c66e28896c3b1" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33885 } }
I20260618 22:31:40.484086 29349 ts_tablet_manager.cc:1428] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260618 22:31:40.483995 29353 consensus_queue.cc:260] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c5c1dca7054362bc660f01ac1e25ce" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43575 } } peers { permanent_uuid: "222eb63b4bd2460bbb8337d7efd5449e" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36923 } } peers { permanent_uuid: "4497eb46020e4844ba1c66e28896c3b1" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33885 } }
I20260618 22:31:40.485126 29331 heartbeater.cc:503] Master 127.10.242.254:45687 was elected leader, sending a full tablet report...
I20260618 22:31:40.485582 29201 heartbeater.cc:503] Master 127.10.242.254:45687 was elected leader, sending a full tablet report...
I20260618 22:31:40.486233 29353 ts_tablet_manager.cc:1428] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260618 22:31:40.487490 29350 ts_tablet_manager.cc:1428] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
W20260618 22:31:40.491104 29071 tablet.cc:2369] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:31:40.583436 29355 raft_consensus.cc:482] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:31:40.583760 29355 raft_consensus.cc:504] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c5c1dca7054362bc660f01ac1e25ce" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43575 } } peers { permanent_uuid: "222eb63b4bd2460bbb8337d7efd5449e" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36923 } } peers { permanent_uuid: "4497eb46020e4844ba1c66e28896c3b1" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33885 } }
I20260618 22:31:40.585718 29355 leader_election.cc:288] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9c5c1dca7054362bc660f01ac1e25ce (127.10.242.195:43575), 4497eb46020e4844ba1c66e28896c3b1 (127.10.242.194:33885)
I20260618 22:31:40.592399 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:40.586130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:58741 (local address 127.10.242.195:43575)
0618 22:31:40.586338 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:40.586344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:40.586936 (+ 592us) server_negotiation.cc:407] Connection header received
0618 22:31:40.587116 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:40.587122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:40.587192 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:40.587256 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:40.587869 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.588599 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.590802 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.591556 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.591688 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:40.591759 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:40.592035 (+ 276us) server_negotiation.cc:299] Negotiation successful
0618 22:31:40.592162 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:31:40.592427 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:40.586248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:58397 (local address 127.10.242.194:33885)
0618 22:31:40.586411 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:40.586418 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:40.587017 (+ 599us) server_negotiation.cc:407] Connection header received
0618 22:31:40.587165 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:40.587169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:40.587240 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:40.587327 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:40.588006 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.588706 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.590819 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.591571 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.591711 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:40.591777 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:40.592032 (+ 255us) server_negotiation.cc:299] Negotiation successful
0618 22:31:40.592155 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260618 22:31:40.592535 29359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:40.586144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:33885 (local address 127.10.242.193:58397)
0618 22:31:40.586906 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:40.586924 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:40.587049 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:40.587353 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:40.587358 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:40.587385 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:40.587822 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:40.587835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.588847 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.588851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:40.590617 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:40.590629 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.591751 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.591757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:40.591890 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:40.591913 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:31:40.591992 (+ 79us) client_negotiation.cc:240] Negotiation successful
0618 22:31:40.592161 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"mutex_wait_us":161,"thread_start_us":401,"threads_started":1}
I20260618 22:31:40.592561 29358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:40.585966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:43575 (local address 127.10.242.193:58741)
0618 22:31:40.586821 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:40.586837 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:40.586968 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:40.587294 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:40.587300 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:40.587324 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:40.587721 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:40.587733 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.588740 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.588746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:40.590611 (+ 1865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:40.590624 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.591738 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.591744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:40.591895 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:40.591917 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:31:40.591992 (+ 75us) client_negotiation.cc:240] Negotiation successful
0618 22:31:40.592152 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"mutex_wait_us":114,"thread_start_us":506,"threads_started":1}
I20260618 22:31:40.593528 29156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e36ddd84dc04ff98b89391c5d5366ba" candidate_uuid: "222eb63b4bd2460bbb8337d7efd5449e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4497eb46020e4844ba1c66e28896c3b1" is_pre_election: true
I20260618 22:31:40.593775 29286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e36ddd84dc04ff98b89391c5d5366ba" candidate_uuid: "222eb63b4bd2460bbb8337d7efd5449e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9c5c1dca7054362bc660f01ac1e25ce" is_pre_election: true
I20260618 22:31:40.594030 29156 raft_consensus.cc:2454] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222eb63b4bd2460bbb8337d7efd5449e in term 0.
I20260618 22:31:40.594228 29286 raft_consensus.cc:2454] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222eb63b4bd2460bbb8337d7efd5449e in term 0.
I20260618 22:31:40.594668 28958 leader_election.cc:302] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 222eb63b4bd2460bbb8337d7efd5449e, 4497eb46020e4844ba1c66e28896c3b1; no voters:
I20260618 22:31:40.595052 29355 raft_consensus.cc:2781] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:31:40.595221 29355 raft_consensus.cc:482] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:31:40.595320 29355 raft_consensus.cc:3037] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:40.597443 29355 raft_consensus.cc:504] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c5c1dca7054362bc660f01ac1e25ce" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43575 } } peers { permanent_uuid: "222eb63b4bd2460bbb8337d7efd5449e" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36923 } } peers { permanent_uuid: "4497eb46020e4844ba1c66e28896c3b1" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33885 } }
I20260618 22:31:40.598013 29355 leader_election.cc:288] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [CANDIDATE]: Term 1 election: Requested vote from peers d9c5c1dca7054362bc660f01ac1e25ce (127.10.242.195:43575), 4497eb46020e4844ba1c66e28896c3b1 (127.10.242.194:33885)
I20260618 22:31:40.598541 29286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e36ddd84dc04ff98b89391c5d5366ba" candidate_uuid: "222eb63b4bd2460bbb8337d7efd5449e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9c5c1dca7054362bc660f01ac1e25ce"
I20260618 22:31:40.598543 29156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e36ddd84dc04ff98b89391c5d5366ba" candidate_uuid: "222eb63b4bd2460bbb8337d7efd5449e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4497eb46020e4844ba1c66e28896c3b1"
I20260618 22:31:40.598754 29286 raft_consensus.cc:3037] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:40.598754 29156 raft_consensus.cc:3037] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:40.601976 29156 raft_consensus.cc:2454] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222eb63b4bd2460bbb8337d7efd5449e in term 1.
I20260618 22:31:40.601974 29286 raft_consensus.cc:2454] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222eb63b4bd2460bbb8337d7efd5449e in term 1.
I20260618 22:31:40.602532 28958 leader_election.cc:302] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 222eb63b4bd2460bbb8337d7efd5449e, 4497eb46020e4844ba1c66e28896c3b1, d9c5c1dca7054362bc660f01ac1e25ce; no voters:
I20260618 22:31:40.602887 29355 raft_consensus.cc:2781] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:31:40.603356 29355 raft_consensus.cc:686] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [term 1 LEADER]: Becoming Leader. State: Replica: 222eb63b4bd2460bbb8337d7efd5449e, State: Running, Role: LEADER
I20260618 22:31:40.603812 29355 consensus_queue.cc:237] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c5c1dca7054362bc660f01ac1e25ce" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43575 } } peers { permanent_uuid: "222eb63b4bd2460bbb8337d7efd5449e" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36923 } } peers { permanent_uuid: "4497eb46020e4844ba1c66e28896c3b1" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33885 } }
I20260618 22:31:40.608459 28883 catalog_manager.cc:5495] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e reported cstate change: term changed from 0 to 1, leader changed from <none> to 222eb63b4bd2460bbb8337d7efd5449e (127.10.242.193). New cstate: current_term: 1 leader_uuid: "222eb63b4bd2460bbb8337d7efd5449e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c5c1dca7054362bc660f01ac1e25ce" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222eb63b4bd2460bbb8337d7efd5449e" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 36923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4497eb46020e4844ba1c66e28896c3b1" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33885 } health_report { overall_health: UNKNOWN } } }
W20260618 22:31:40.630584 29332 tablet.cc:2369] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:31:40.634575 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:40.629044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45566 (local address 127.10.242.193:36923)
0618 22:31:40.629257 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:40.629264 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:40.629338 (+ 74us) server_negotiation.cc:407] Connection header received
0618 22:31:40.629575 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:40.629580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:40.629670 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:40.629822 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:31:40.630221 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.631022 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.632417 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.632815 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.632927 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:40.633460 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:31:40.634014 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:31:40.634145 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:40.634220 (+ 75us) server_negotiation.cc:299] Negotiation successful
0618 22:31:40.634300 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260618 22:31:40.642769 29286 raft_consensus.cc:1261] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce [term 1 FOLLOWER]: Refusing update from remote peer 222eb63b4bd2460bbb8337d7efd5449e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:31:40.642767 29156 raft_consensus.cc:1261] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1 [term 1 FOLLOWER]: Refusing update from remote peer 222eb63b4bd2460bbb8337d7efd5449e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:31:40.643719 29360 consensus_queue.cc:1035] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9c5c1dca7054362bc660f01ac1e25ce" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 43575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:31:40.644129 29355 consensus_queue.cc:1035] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4497eb46020e4844ba1c66e28896c3b1" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 33885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:31:40.655966 29365 mvcc.cc:205] Tried to move back new op lower bound from 7298342505022590976 to 7298342504876949504. Current Snapshot: MvccSnapshot[applied={T|T < 7298342505022590976}]
I20260618 22:31:40.656208 29363 mvcc.cc:205] Tried to move back new op lower bound from 7298342505022590976 to 7298342504876949504. Current Snapshot: MvccSnapshot[applied={T|T < 7298342505022590976}]
I20260618 22:31:40.669061 28883 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:31:40.669411 28883 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:31:40.673429 28883 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 647a435ebda94778b7ee7d499083b887: Sending DeleteTablet for 3 replicas of tablet 7e36ddd84dc04ff98b89391c5d5366ba
I20260618 22:31:40.674600 29136 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e36ddd84dc04ff98b89391c5d5366ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:31:40 UTC) from {username='slave'} at 127.0.0.1:39810
I20260618 22:31:40.674607 29005 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e36ddd84dc04ff98b89391c5d5366ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:31:40 UTC) from {username='slave'} at 127.0.0.1:45554
I20260618 22:31:40.674600 29266 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e36ddd84dc04ff98b89391c5d5366ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:31:40 UTC) from {username='slave'} at 127.0.0.1:55980
I20260618 22:31:40.684428 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 28943
I20260618 22:31:40.691820 29375 tablet_replica.cc:330] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e: stopping tablet replica
I20260618 22:31:40.692521 29375 raft_consensus.cc:2229] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:31:40.693387 29375 raft_consensus.cc:2258] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:31:40.696151 29375 ts_tablet_manager.cc:1905] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:31:40.699234 29374 tablet_replica.cc:330] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1: stopping tablet replica
I20260618 22:31:40.699910 29374 raft_consensus.cc:2229] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:31:40.700568 29374 raft_consensus.cc:2258] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:31:40.704830 29375 ts_tablet_manager.cc:1918] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:31:40.705133 29375 log.cc:1196] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/7e36ddd84dc04ff98b89391c5d5366ba
I20260618 22:31:40.705832 29375 ts_tablet_manager.cc:1939] T 7e36ddd84dc04ff98b89391c5d5366ba P 222eb63b4bd2460bbb8337d7efd5449e: Deleting consensus metadata
I20260618 22:31:40.706976 29374 ts_tablet_manager.cc:1905] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:31:40.707856 28870 catalog_manager.cc:4841] TS 222eb63b4bd2460bbb8337d7efd5449e (127.10.242.193:36923): tablet 7e36ddd84dc04ff98b89391c5d5366ba (table test-table [id=5f37f315103940df830f5c30576f772c]) successfully deleted
I20260618 22:31:40.708032 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:40.686184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45578 (local address 127.10.242.193:36923)
0618 22:31:40.686428 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:40.686436 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:40.686761 (+ 325us) server_negotiation.cc:407] Connection header received
0618 22:31:40.687031 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:40.687038 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:40.687163 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:40.687280 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:40.694441 (+ 7161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.695722 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.700814 (+ 5092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.701818 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.701968 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:40.704168 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:40.704180 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:40.704203 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:40.705165 (+ 962us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:40.705250 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:40.705285 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:40.705390 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:40.707471 (+ 2081us) server_negotiation.cc:299] Negotiation successful
0618 22:31:40.707618 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260618 22:31:40.709334 29065 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:40.710691 29373 tablet_replica.cc:330] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce: stopping tablet replica
I20260618 22:31:40.711283 29373 raft_consensus.cc:2229] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:31:40.713593 29373 raft_consensus.cc:2258] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:31:40.716008 29373 ts_tablet_manager.cc:1905] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:31:40.718631 29374 ts_tablet_manager.cc:1918] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:31:40.718823 29374 log.cc:1196] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/7e36ddd84dc04ff98b89391c5d5366ba
I20260618 22:31:40.719391 29374 ts_tablet_manager.cc:1939] T 7e36ddd84dc04ff98b89391c5d5366ba P 4497eb46020e4844ba1c66e28896c3b1: Deleting consensus metadata
I20260618 22:31:40.721542 28868 catalog_manager.cc:4841] TS 4497eb46020e4844ba1c66e28896c3b1 (127.10.242.194:33885): tablet 7e36ddd84dc04ff98b89391c5d5366ba (table test-table [id=5f37f315103940df830f5c30576f772c]) successfully deleted
I20260618 22:31:40.722733 29373 ts_tablet_manager.cc:1918] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:31:40.722890 29373 log.cc:1196] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/7e36ddd84dc04ff98b89391c5d5366ba
I20260618 22:31:40.723353 29373 ts_tablet_manager.cc:1939] T 7e36ddd84dc04ff98b89391c5d5366ba P d9c5c1dca7054362bc660f01ac1e25ce: Deleting consensus metadata
I20260618 22:31:40.725010 28871 catalog_manager.cc:4841] TS d9c5c1dca7054362bc660f01ac1e25ce (127.10.242.195:43575): tablet 7e36ddd84dc04ff98b89391c5d5366ba (table test-table [id=5f37f315103940df830f5c30576f772c]) successfully deleted
I20260618 22:31:40.807251 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 28943
I20260618 22:31:40.822643 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 29074
I20260618 22:31:40.829583 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:40.823442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39816 (local address 127.10.242.194:33885)
0618 22:31:40.823618 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:40.823623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:40.823645 (+ 22us) server_negotiation.cc:407] Connection header received
0618 22:31:40.823758 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:40.823764 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:40.823869 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:40.823969 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:40.824306 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.825124 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.826514 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.826930 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.827038 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:40.828123 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:40.828131 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:40.828146 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:40.828753 (+ 607us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:40.828802 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:40.828823 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:40.828914 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:40.829154 (+ 240us) server_negotiation.cc:299] Negotiation successful
0618 22:31:40.829289 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:31:40.830232 29196 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:40.921684 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 29074
I20260618 22:31:40.935458 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 29204
I20260618 22:31:40.942207 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:40.936222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55986 (local address 127.10.242.195:43575)
0618 22:31:40.936383 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:40.936388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:40.936539 (+ 151us) server_negotiation.cc:407] Connection header received
0618 22:31:40.936695 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:40.936699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:40.936797 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:40.936890 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:40.937324 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.938090 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.939370 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:40.939762 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:40.939866 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:40.940862 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:40.940871 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:40.940886 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:40.941557 (+ 671us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:40.941607 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:40.941627 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:40.941717 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:40.941831 (+ 114us) server_negotiation.cc:299] Negotiation successful
0618 22:31:40.941920 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260618 22:31:40.942821 29326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:41.041613 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 29204
I20260618 22:31:41.054616 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 28853
I20260618 22:31:41.055743 28914 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:41.183090 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 28853
2026-06-18T22:31:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2722 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:31:41 dist-test-slave-0956 krb5kdc[29383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:31:41 dist-test-slave-0956 krb5kdc[29383](info): set up 2 sockets
Jun 18 22:31:41 dist-test-slave-0956 krb5kdc[29383](info): commencing operation
krb5kdc: starting...
W20260618 22:31:42.196854 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.977s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:31:42 dist-test-slave-0956 krb5kdc[29383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:31:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:31:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:31:42.431806 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:38185
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:38185
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:31:42.755671 29396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:42.755990 29396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:42.756127 29396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:42.763437 29396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:31:42.763564 29396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:42.763633 29396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:42.763713 29396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:31:42.763775 29396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:31:42.773669 29396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:38185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:38185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:42.774979 29396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:42.776279 29396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:42.788130 29401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:42.788115 29404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:42.788084 29402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:42.789022 29396 server_base.cc:1034] running on GCE node
I20260618 22:31:42.789844 29396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:42.791253 29396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:42.792450 29396 hybrid_clock.cc:648] HybridClock initialized: now 1781821902792400 us; error 50 us; skew 500 ppm
I20260618 22:31:42.792811 29396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:31:42 dist-test-slave-0956 krb5kdc[29383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821902, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:31:42.795861 29396 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:31:42.797497 29396 webserver.cc:460] Webserver started at http://127.10.242.254:35675/ using document root <none> and password file <none>
I20260618 22:31:42.798173 29396 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:42.798280 29396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:42.798565 29396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:42.801571 29396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "04e62df9e8b14bff80f1e8db7230ba56"
format_stamp: "Formatted at 2026-06-18 22:31:42 on dist-test-slave-0956"
server_key: "ed05cd1c471309d43691d733af0a452c"
server_key_iv: "9667af38cfd57cc9d2cbc0bf7c405b49"
server_key_version: "encryptionkey@0"
I20260618 22:31:42.802338 29396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "04e62df9e8b14bff80f1e8db7230ba56"
format_stamp: "Formatted at 2026-06-18 22:31:42 on dist-test-slave-0956"
server_key: "ed05cd1c471309d43691d733af0a452c"
server_key_iv: "9667af38cfd57cc9d2cbc0bf7c405b49"
server_key_version: "encryptionkey@0"
I20260618 22:31:42.807492 29396 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:31:42.810974 29411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:42.812386 29396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:31:42.812546 29396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "04e62df9e8b14bff80f1e8db7230ba56"
format_stamp: "Formatted at 2026-06-18 22:31:42 on dist-test-slave-0956"
server_key: "ed05cd1c471309d43691d733af0a452c"
server_key_iv: "9667af38cfd57cc9d2cbc0bf7c405b49"
server_key_version: "encryptionkey@0"
I20260618 22:31:42.812732 29396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:42.840988 29396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:42.845467 29396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:42.845789 29396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:42.865283 29396 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:38185
I20260618 22:31:42.865289 29462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:38185 every 8 connection(s)
I20260618 22:31:42.866743 29396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:31:42.870858 29463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:42.874329 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 29396
I20260618 22:31:42.874508 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:31:42.874948 11211 external_mini_cluster.cc:1428] Setting key c72fe7366d3923fe1cbbfd1985206f06
I20260618 22:31:42.884513 29463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56: Bootstrap starting.
I20260618 22:31:42.888214 29463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:42.889283 29463 log.cc:824] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:31:42 dist-test-slave-0956 krb5kdc[29383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:31:42.892524 29463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56: No bootstrap required, opened a new log
I20260618 22:31:42.898991 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:42.876956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48602 (local address 127.10.242.254:38185)
0618 22:31:42.877828 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:42.877845 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:42.877899 (+ 54us) server_negotiation.cc:407] Connection header received
0618 22:31:42.879997 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:42.880031 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:42.880653 (+ 622us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:42.881171 (+ 518us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:42.882528 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:42.884521 (+ 1993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:42.885942 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:42.887033 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:42.887217 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:42.891513 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:42.891522 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:42.891537 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:42.891574 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:42.894627 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:42.895581 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:42.895587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:42.895597 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:42.895787 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:42.896185 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:42.896188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:42.896193 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:42.896753 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:42.896971 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:42.897493 (+ 522us) server_negotiation.cc:299] Negotiation successful
0618 22:31:42.897911 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":355,"threads_started":1}
I20260618 22:31:42.901273 29463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e62df9e8b14bff80f1e8db7230ba56" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 38185 } }
I20260618 22:31:42.901655 29463 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:42.901757 29463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04e62df9e8b14bff80f1e8db7230ba56, State: Initialized, Role: FOLLOWER
I20260618 22:31:42.902283 29463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e62df9e8b14bff80f1e8db7230ba56" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 38185 } }
I20260618 22:31:42.902508 29463 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:31:42.902621 29463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:31:42.902762 29463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:42.904914 29463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e62df9e8b14bff80f1e8db7230ba56" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 38185 } }
I20260618 22:31:42.905392 29463 leader_election.cc:302] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04e62df9e8b14bff80f1e8db7230ba56; no voters:
I20260618 22:31:42.905957 29463 leader_election.cc:288] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:31:42.906093 29468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:31:42.907994 29463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:31:42.909404 29468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56 [term 1 LEADER]: Becoming Leader. State: Replica: 04e62df9e8b14bff80f1e8db7230ba56, State: Running, Role: LEADER
I20260618 22:31:42.910151 29468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e62df9e8b14bff80f1e8db7230ba56" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 38185 } }
I20260618 22:31:42.913280 29470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04e62df9e8b14bff80f1e8db7230ba56. Latest consensus state: current_term: 1 leader_uuid: "04e62df9e8b14bff80f1e8db7230ba56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e62df9e8b14bff80f1e8db7230ba56" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 38185 } } }
I20260618 22:31:42.913404 29469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04e62df9e8b14bff80f1e8db7230ba56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e62df9e8b14bff80f1e8db7230ba56" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 38185 } } }
I20260618 22:31:42.913805 29470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56 [sys.catalog]: This master's current role is: LEADER
I20260618 22:31:42.913956 29469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56 [sys.catalog]: This master's current role is: LEADER
W20260618 22:31:42.921674 29482 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:31:42.921799 29482 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:31:42.921979 29483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:31:42.923555 29483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:31:42.931348 29483 catalog_manager.cc:1347] Generated new cluster ID: 018008033a6e4418a46a6879272f7db4
I20260618 22:31:42.931470 29483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:31:42.940796 29483 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:31:42.942121 29483 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:31:42.958703 29483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56: Generated new TSK 0
I20260618 22:31:42.959585 29483 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
I20260618 22:31:43.020411 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:38185
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:31:43.348363 29489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:43.348649 29489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:43.348773 29489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:43.356019 29489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:43.356154 29489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:43.356340 29489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:31:43.366312 29489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:38185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:43.367628 29489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:43.368939 29489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:43.381803 29495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:43.381831 29497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:43.381769 29494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:43.382598 29489 server_base.cc:1034] running on GCE node
I20260618 22:31:43.383206 29489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:43.384186 29489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:43.385406 29489 hybrid_clock.cc:648] HybridClock initialized: now 1781821903385355 us; error 40 us; skew 500 ppm
I20260618 22:31:43.385756 29489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:31:43 dist-test-slave-0956 krb5kdc[29383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821903, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:31:43.388913 29489 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:31:43.390369 29489 webserver.cc:460] Webserver started at http://127.10.242.193:34253/ using document root <none> and password file <none>
I20260618 22:31:43.391072 29489 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:43.391185 29489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:43.391479 29489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:43.394418 29489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "e2bb454d1d9144588f883147d7ec1d2c"
format_stamp: "Formatted at 2026-06-18 22:31:43 on dist-test-slave-0956"
server_key: "bdef0cc4677f2403c99359fb79996b56"
server_key_iv: "1470e6dd42449aeeec7475456ecc5544"
server_key_version: "encryptionkey@0"
I20260618 22:31:43.395167 29489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "e2bb454d1d9144588f883147d7ec1d2c"
format_stamp: "Formatted at 2026-06-18 22:31:43 on dist-test-slave-0956"
server_key: "bdef0cc4677f2403c99359fb79996b56"
server_key_iv: "1470e6dd42449aeeec7475456ecc5544"
server_key_version: "encryptionkey@0"
I20260618 22:31:43.400228 29489 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260618 22:31:43.404078 29504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:43.405450 29489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:31:43.405593 29489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "e2bb454d1d9144588f883147d7ec1d2c"
format_stamp: "Formatted at 2026-06-18 22:31:43 on dist-test-slave-0956"
server_key: "bdef0cc4677f2403c99359fb79996b56"
server_key_iv: "1470e6dd42449aeeec7475456ecc5544"
server_key_version: "encryptionkey@0"
I20260618 22:31:43.405782 29489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:43.443938 29489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:43.448104 29489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:43.448393 29489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:43.449394 29489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:43.450758 29489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:43.450841 29489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:43.450959 29489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:43.451011 29489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:43.481283 29489 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:35055
I20260618 22:31:43.481326 29616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:35055 every 8 connection(s)
I20260618 22:31:43.482574 29489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:31:43.486586 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 29489
I20260618 22:31:43.486778 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:31:43.487327 11211 external_mini_cluster.cc:1428] Setting key 97c526ee4d550e29e3b973d153b3417c
Jun 18 22:31:43 dist-test-slave-0956 krb5kdc[29383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821903, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:31:43.503152 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:43.485262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:42135 (local address 127.10.242.254:38185)
0618 22:31:43.485688 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:43.485693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:43.487177 (+ 1484us) server_negotiation.cc:407] Connection header received
0618 22:31:43.488871 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:43.488875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:43.488943 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:43.489056 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:43.490653 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:43.491664 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:43.493351 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:43.493882 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:43.493998 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:43.497516 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:43.497521 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:43.497524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:43.497565 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:43.499997 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:43.500734 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:43.500739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:43.500741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:43.500871 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:43.501356 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:43.501359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:43.501361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:43.501648 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:43.501780 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:43.502533 (+ 753us) server_negotiation.cc:299] Negotiation successful
0618 22:31:43.502793 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":194,"threads_started":1}
I20260618 22:31:43.504189 29620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:43.485869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:38185 (local address 127.10.242.193:42135)
0618 22:31:43.486612 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:43.486653 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:43.488589 (+ 1936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:43.489354 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:43.489367 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:43.489887 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:31:43.490470 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:43.490489 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:43.491836 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:43.491840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:43.493219 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:43.493232 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:43.494111 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:43.494115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:43.494309 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:43.494950 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:31:43.494979 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:31:43.497291 (+ 2312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:31:43.500141 (+ 2850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:31:43.500151 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:31:43.500169 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:31:43.500608 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:31:43.500976 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:31:43.500982 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:31:43.500986 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:31:43.501244 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:31:43.501769 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:31:43.501777 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:31:43.502148 (+ 371us) client_negotiation.cc:769] Sending connection context
0618 22:31:43.502393 (+ 245us) client_negotiation.cc:240] Negotiation successful
0618 22:31:43.502948 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":305,"threads_started":1}
I20260618 22:31:43.505949 29617 heartbeater.cc:347] Connected to a master server at 127.10.242.254:38185
I20260618 22:31:43.506299 29617 heartbeater.cc:464] Registering TS with master...
I20260618 22:31:43.507025 29617 heartbeater.cc:511] Master 127.10.242.254:38185 requested a full tablet report, sending...
I20260618 22:31:43.509568 29428 ts_manager.cc:194] Registered new tserver with Master: e2bb454d1d9144588f883147d7ec1d2c (127.10.242.193:35055)
WARNING: no policy specified for kudu/127.10.242.194@KRBTEST.COM; defaulting to no policy
I20260618 22:31:43.511379 29428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:42135
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
I20260618 22:31:43.540911 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:38185
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:31:43.862401 29623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:43.862706 29623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:43.862825 29623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:43.870286 29623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:43.870407 29623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:43.870560 29623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:31:43.880462 29623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:38185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:43.881846 29623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:43.883167 29623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:43.895923 29628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:43.895905 29631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:43.896039 29629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:43.896591 29623 server_base.cc:1034] running on GCE node
I20260618 22:31:43.897282 29623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:43.898236 29623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:43.899454 29623 hybrid_clock.cc:648] HybridClock initialized: now 1781821903899404 us; error 46 us; skew 500 ppm
I20260618 22:31:43.899776 29623 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:31:43 dist-test-slave-0956 krb5kdc[29383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821903, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:31:43.902874 29623 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:31:43.904291 29623 webserver.cc:460] Webserver started at http://127.10.242.194:42141/ using document root <none> and password file <none>
I20260618 22:31:43.904968 29623 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:43.905066 29623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:43.905489 29623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:43.908505 29623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "7381d19ef72743d6959c83e501f6be2b"
format_stamp: "Formatted at 2026-06-18 22:31:43 on dist-test-slave-0956"
server_key: "d00abcef50ac265c85cd7a5da38dae9a"
server_key_iv: "f51b0d5362a456ea293639a3e5887adc"
server_key_version: "encryptionkey@0"
I20260618 22:31:43.909389 29623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "7381d19ef72743d6959c83e501f6be2b"
format_stamp: "Formatted at 2026-06-18 22:31:43 on dist-test-slave-0956"
server_key: "d00abcef50ac265c85cd7a5da38dae9a"
server_key_iv: "f51b0d5362a456ea293639a3e5887adc"
server_key_version: "encryptionkey@0"
I20260618 22:31:43.914776 29623 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:31:43.918725 29638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:43.920341 29623 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:31:43.920500 29623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "7381d19ef72743d6959c83e501f6be2b"
format_stamp: "Formatted at 2026-06-18 22:31:43 on dist-test-slave-0956"
server_key: "d00abcef50ac265c85cd7a5da38dae9a"
server_key_iv: "f51b0d5362a456ea293639a3e5887adc"
server_key_version: "encryptionkey@0"
I20260618 22:31:43.920696 29623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:43.938814 29623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:43.942991 29623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:43.943320 29623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:43.944314 29623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:43.945739 29623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:43.945818 29623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:43.945943 29623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:43.945995 29623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:43.976686 29623 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:36659
I20260618 22:31:43.976727 29750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:36659 every 8 connection(s)
I20260618 22:31:43.977979 29623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:31:43.984251 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 29623
I20260618 22:31:43.984440 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:31:43.984992 11211 external_mini_cluster.cc:1428] Setting key fa2096c57a860c76afe7507789a784b0
Jun 18 22:31:43 dist-test-slave-0956 krb5kdc[29383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821903, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:31:43.999778 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:43.980548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:49269 (local address 127.10.242.254:38185)
0618 22:31:43.980773 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:43.980777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:43.981979 (+ 1202us) server_negotiation.cc:407] Connection header received
0618 22:31:43.983765 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:43.983770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:43.983847 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:43.983980 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:43.986023 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:43.986745 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:43.988939 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:43.989452 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:43.989588 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:43.993645 (+ 4057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:43.993650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:43.993654 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:43.993692 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:43.996242 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:43.997127 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:43.997140 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:43.997143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:43.997295 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:43.997782 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:43.997787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:43.997790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:43.998166 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:43.998303 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:43.999226 (+ 923us) server_negotiation.cc:299] Negotiation successful
0618 22:31:43.999459 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260618 22:31:44.000805 29753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:43.981071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:38185 (local address 127.10.242.194:49269)
0618 22:31:43.981818 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:43.981856 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:43.983468 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:43.984254 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:43.984266 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:43.985032 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:31:43.985834 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:43.985857 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:43.986898 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:43.986902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:43.988762 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:43.988780 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:43.989721 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:43.989728 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:43.990022 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:43.990764 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:31:43.990799 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:31:43.993363 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:31:43.996414 (+ 3051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:31:43.996424 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:31:43.996446 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:31:43.996937 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:31:43.997428 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:31:43.997432 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:31:43.997434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:31:43.997656 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:31:43.998319 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:31:43.998326 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:31:43.998802 (+ 476us) client_negotiation.cc:769] Sending connection context
0618 22:31:43.999109 (+ 307us) client_negotiation.cc:240] Negotiation successful
0618 22:31:43.999548 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":270,"threads_started":1}
I20260618 22:31:44.002380 29751 heartbeater.cc:347] Connected to a master server at 127.10.242.254:38185
I20260618 22:31:44.002723 29751 heartbeater.cc:464] Registering TS with master...
I20260618 22:31:44.003582 29751 heartbeater.cc:511] Master 127.10.242.254:38185 requested a full tablet report, sending...
I20260618 22:31:44.005367 29428 ts_manager.cc:194] Registered new tserver with Master: 7381d19ef72743d6959c83e501f6be2b (127.10.242.194:36659)
I20260618 22:31:44.006207 29428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:49269
WARNING: no policy specified for kudu/127.10.242.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
I20260618 22:31:44.038574 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:38185
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:31:44.358212 29756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:44.358513 29756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:44.358659 29756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:44.365954 29756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:44.366096 29756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:44.366246 29756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:31:44.375979 29756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:38185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:44.377344 29756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:44.378763 29756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:44.391774 29761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:44.391747 29762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:44.391952 29764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:44.392472 29756 server_base.cc:1034] running on GCE node
I20260618 22:31:44.393044 29756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:44.393999 29756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:44.395217 29756 hybrid_clock.cc:648] HybridClock initialized: now 1781821904395167 us; error 39 us; skew 500 ppm
I20260618 22:31:44.395517 29756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:31:44 dist-test-slave-0956 krb5kdc[29383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821904, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:31:44.398509 29756 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:31:44.399996 29756 webserver.cc:460] Webserver started at http://127.10.242.195:40601/ using document root <none> and password file <none>
I20260618 22:31:44.400664 29756 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:44.400779 29756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:44.401067 29756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:44.404079 29756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "aa4b7fe00f66474d9440aca38859b163"
format_stamp: "Formatted at 2026-06-18 22:31:44 on dist-test-slave-0956"
server_key: "79141199e952e0a925f5e3ab45c6547b"
server_key_iv: "4e0daf2a2f45f7abfd42ad7d79e9578c"
server_key_version: "encryptionkey@0"
I20260618 22:31:44.404852 29756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "aa4b7fe00f66474d9440aca38859b163"
format_stamp: "Formatted at 2026-06-18 22:31:44 on dist-test-slave-0956"
server_key: "79141199e952e0a925f5e3ab45c6547b"
server_key_iv: "4e0daf2a2f45f7abfd42ad7d79e9578c"
server_key_version: "encryptionkey@0"
I20260618 22:31:44.410434 29756 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260618 22:31:44.413951 29771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:44.415329 29756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:31:44.415477 29756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "aa4b7fe00f66474d9440aca38859b163"
format_stamp: "Formatted at 2026-06-18 22:31:44 on dist-test-slave-0956"
server_key: "79141199e952e0a925f5e3ab45c6547b"
server_key_iv: "4e0daf2a2f45f7abfd42ad7d79e9578c"
server_key_version: "encryptionkey@0"
I20260618 22:31:44.415680 29756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:44.428615 29756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:44.432323 29756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:44.432585 29756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:44.433504 29756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:44.434777 29756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:44.434850 29756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:44.434964 29756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:44.435016 29756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:44.464977 29756 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:40053
I20260618 22:31:44.465016 29883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:40053 every 8 connection(s)
I20260618 22:31:44.466245 29756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:31:44.470731 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 29756
I20260618 22:31:44.470903 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:31:44.471339 11211 external_mini_cluster.cc:1428] Setting key 533e3bb3c378ca830fdfc9816fec7e51
Jun 18 22:31:44 dist-test-slave-0956 krb5kdc[29383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821904, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:31:44.486038 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:44.468750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:38141 (local address 127.10.242.254:38185)
0618 22:31:44.468928 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:44.468932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:44.470107 (+ 1175us) server_negotiation.cc:407] Connection header received
0618 22:31:44.472117 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:44.472124 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:44.472217 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:44.472321 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:44.474168 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.474945 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.476454 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.476896 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.477007 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:44.480334 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:44.480339 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:44.480342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:44.480378 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:44.482904 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:44.483615 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:44.483619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:44.483621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:44.483744 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:44.484153 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:44.484156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:44.484158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:44.484423 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:44.484567 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:44.485511 (+ 944us) server_negotiation.cc:299] Negotiation successful
0618 22:31:44.485716 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260618 22:31:44.486999 29886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:44.469246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:38185 (local address 127.10.242.195:38141)
0618 22:31:44.469937 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:44.469980 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:44.471778 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:44.472594 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:44.472606 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:44.473343 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:31:44.474004 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:44.474025 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.475077 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.475081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:44.476310 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:44.476322 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.477071 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.477076 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:44.477315 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:44.477930 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:31:44.477952 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:31:44.480110 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:31:44.483049 (+ 2939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:31:44.483058 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:31:44.483074 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:31:44.483493 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:31:44.483843 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:31:44.483848 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:31:44.483852 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:31:44.484040 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:31:44.484548 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:31:44.484556 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:31:44.484963 (+ 407us) client_negotiation.cc:769] Sending connection context
0618 22:31:44.485256 (+ 293us) client_negotiation.cc:240] Negotiation successful
0618 22:31:44.485795 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":267,"threads_started":1}
I20260618 22:31:44.488179 29884 heartbeater.cc:347] Connected to a master server at 127.10.242.254:38185
I20260618 22:31:44.488464 29884 heartbeater.cc:464] Registering TS with master...
I20260618 22:31:44.489199 29884 heartbeater.cc:511] Master 127.10.242.254:38185 requested a full tablet report, sending...
I20260618 22:31:44.490509 29428 ts_manager.cc:194] Registered new tserver with Master: aa4b7fe00f66474d9440aca38859b163 (127.10.242.195:40053)
I20260618 22:31:44.491325 29428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:38141
I20260618 22:31:44.498859 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:31:44.514604 29617 heartbeater.cc:503] Master 127.10.242.254:38185 was elected leader, sending a full tablet report...
I20260618 22:31:44.516381 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:44.502694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48606 (local address 127.10.242.254:38185)
0618 22:31:44.502849 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:44.502853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:44.503202 (+ 349us) server_negotiation.cc:407] Connection header received
0618 22:31:44.503334 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:44.503338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:44.503396 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:44.503476 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:44.504827 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.505585 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.506988 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.507389 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.507475 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:44.510570 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:44.510577 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:44.510583 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:44.510626 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:44.513372 (+ 2746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:44.514170 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:44.514174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:44.514176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:44.514312 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:44.514857 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:44.514862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:44.514864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:44.515285 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:44.515532 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:44.515840 (+ 308us) server_negotiation.cc:299] Negotiation successful
0618 22:31:44.516050 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:31:44.520501 29428 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48606:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:31:44.527171 29428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:31:44.559633 29900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:44.541243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:36659 (local address 127.0.0.1:41690)
0618 22:31:44.552119 (+ 10876us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:44.552137 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:44.552275 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:44.553203 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:44.553207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:44.553235 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:44.553592 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:44.553604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.555228 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.555232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:44.557383 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:44.557394 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.558659 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.558665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:44.558858 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:44.558907 (+ 49us) client_negotiation.cc:769] Sending connection context
0618 22:31:44.559039 (+ 132us) client_negotiation.cc:240] Negotiation successful
0618 22:31:44.559271 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10751,"thread_start_us":11821,"threads_started":1}
I20260618 22:31:44.560058 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:44.539138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:40053 (local address 127.0.0.1:54754)
0618 22:31:44.551758 (+ 12620us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:44.551776 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:44.551903 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:44.552862 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:44.552866 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:44.552890 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:44.553349 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:44.553366 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.554852 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.554857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:44.557209 (+ 2352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:44.557223 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.559194 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.559200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:44.559408 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:44.559464 (+ 56us) client_negotiation.cc:769] Sending connection context
0618 22:31:44.559543 (+ 79us) client_negotiation.cc:240] Negotiation successful
0618 22:31:44.559674 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12450,"mutex_wait_us":113,"thread_start_us":11006,"threads_started":1}
I20260618 22:31:44.560449 29897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:44.539056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:35055 (local address 127.0.0.1:41996)
0618 22:31:44.550555 (+ 11499us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:44.550606 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:44.550832 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:44.552122 (+ 1290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:44.552130 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:44.552162 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:44.552638 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:44.552663 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.554542 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.554547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:44.556892 (+ 2345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:44.556904 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.559091 (+ 2187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.559098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:44.559311 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:44.559868 (+ 557us) client_negotiation.cc:769] Sending connection context
0618 22:31:44.559956 (+ 88us) client_negotiation.cc:240] Negotiation successful
0618 22:31:44.560092 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11237,"mutex_wait_us":622,"thread_start_us":12534,"threads_started":1}
I20260618 22:31:44.560663 29901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:44.541772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41690 (local address 127.10.242.194:36659)
0618 22:31:44.551686 (+ 9914us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:44.551694 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:44.552400 (+ 706us) server_negotiation.cc:407] Connection header received
0618 22:31:44.552504 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:44.552511 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:44.552763 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:44.552939 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:44.553758 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.555041 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.557509 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.558468 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.558623 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:44.558817 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:44.559462 (+ 645us) server_negotiation.cc:299] Negotiation successful
0618 22:31:44.559645 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9770,"thread_start_us":9612,"threads_started":1}
I20260618 22:31:44.561004 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:44.539455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54754 (local address 127.10.242.195:40053)
0618 22:31:44.551336 (+ 11881us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:44.551343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:44.552036 (+ 693us) server_negotiation.cc:407] Connection header received
0618 22:31:44.552143 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:44.552162 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:44.552411 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:44.552611 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:44.553576 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.554712 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.557378 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.558737 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.558911 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:44.559061 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:44.560495 (+ 1434us) server_negotiation.cc:299] Negotiation successful
0618 22:31:44.560690 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11719,"thread_start_us":11548,"threads_started":1}
I20260618 22:31:44.561193 29898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:44.539279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41996 (local address 127.10.242.193:35055)
0618 22:31:44.549747 (+ 10468us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:44.549759 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:44.551066 (+ 1307us) server_negotiation.cc:407] Connection header received
0618 22:31:44.551202 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:44.551211 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:44.551555 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:44.551803 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:44.552847 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.554394 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.557057 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.558520 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.558725 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:44.558928 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:44.560676 (+ 1748us) server_negotiation.cc:299] Negotiation successful
0618 22:31:44.560860 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10206,"thread_start_us":10499,"threads_started":1}
I20260618 22:31:44.564548 29686 tablet_service.cc:1467] Processing CreateTablet for tablet ea3903239bbc4cb4ab06bd3eac6d1137 (DEFAULT_TABLE table=test-table [id=bf64f28653d94011889472fdf06eb1de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:31:44.564630 29552 tablet_service.cc:1467] Processing CreateTablet for tablet ea3903239bbc4cb4ab06bd3eac6d1137 (DEFAULT_TABLE table=test-table [id=bf64f28653d94011889472fdf06eb1de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:31:44.564631 29819 tablet_service.cc:1467] Processing CreateTablet for tablet ea3903239bbc4cb4ab06bd3eac6d1137 (DEFAULT_TABLE table=test-table [id=bf64f28653d94011889472fdf06eb1de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:31:44.565936 29686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea3903239bbc4cb4ab06bd3eac6d1137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:44.566054 29552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea3903239bbc4cb4ab06bd3eac6d1137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:44.566066 29819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea3903239bbc4cb4ab06bd3eac6d1137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:44.577881 29902 tablet_bootstrap.cc:492] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b: Bootstrap starting.
I20260618 22:31:44.581228 29904 tablet_bootstrap.cc:492] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163: Bootstrap starting.
I20260618 22:31:44.581656 29902 tablet_bootstrap.cc:654] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:44.583127 29903 tablet_bootstrap.cc:492] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c: Bootstrap starting.
I20260618 22:31:44.585067 29904 tablet_bootstrap.cc:654] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:44.585240 29902 log.cc:824] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:44.586378 29903 tablet_bootstrap.cc:654] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:44.586985 29904 log.cc:824] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:44.587597 29903 log.cc:824] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:44.589378 29902 tablet_bootstrap.cc:492] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b: No bootstrap required, opened a new log
I20260618 22:31:44.589643 29902 ts_tablet_manager.cc:1397] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260618 22:31:44.590443 29904 tablet_bootstrap.cc:492] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163: No bootstrap required, opened a new log
I20260618 22:31:44.590441 29903 tablet_bootstrap.cc:492] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c: No bootstrap required, opened a new log
I20260618 22:31:44.590708 29903 ts_tablet_manager.cc:1397] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260618 22:31:44.590708 29904 ts_tablet_manager.cc:1397] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260618 22:31:44.597498 29902 raft_consensus.cc:348] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bb454d1d9144588f883147d7ec1d2c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35055 } } peers { permanent_uuid: "aa4b7fe00f66474d9440aca38859b163" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40053 } } peers { permanent_uuid: "7381d19ef72743d6959c83e501f6be2b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36659 } }
I20260618 22:31:44.597895 29902 raft_consensus.cc:374] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:44.597987 29902 raft_consensus.cc:729] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7381d19ef72743d6959c83e501f6be2b, State: Initialized, Role: FOLLOWER
I20260618 22:31:44.598557 29902 consensus_queue.cc:260] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bb454d1d9144588f883147d7ec1d2c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35055 } } peers { permanent_uuid: "aa4b7fe00f66474d9440aca38859b163" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40053 } } peers { permanent_uuid: "7381d19ef72743d6959c83e501f6be2b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36659 } }
I20260618 22:31:44.599606 29751 heartbeater.cc:503] Master 127.10.242.254:38185 was elected leader, sending a full tablet report...
I20260618 22:31:44.601287 29902 ts_tablet_manager.cc:1428] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260618 22:31:44.601112 29904 raft_consensus.cc:348] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bb454d1d9144588f883147d7ec1d2c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35055 } } peers { permanent_uuid: "aa4b7fe00f66474d9440aca38859b163" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40053 } } peers { permanent_uuid: "7381d19ef72743d6959c83e501f6be2b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36659 } }
I20260618 22:31:44.601554 29904 raft_consensus.cc:374] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:44.601675 29904 raft_consensus.cc:729] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa4b7fe00f66474d9440aca38859b163, State: Initialized, Role: FOLLOWER
I20260618 22:31:44.601122 29903 raft_consensus.cc:348] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bb454d1d9144588f883147d7ec1d2c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35055 } } peers { permanent_uuid: "aa4b7fe00f66474d9440aca38859b163" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40053 } } peers { permanent_uuid: "7381d19ef72743d6959c83e501f6be2b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36659 } }
I20260618 22:31:44.601912 29903 raft_consensus.cc:374] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:44.602013 29903 raft_consensus.cc:729] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2bb454d1d9144588f883147d7ec1d2c, State: Initialized, Role: FOLLOWER
I20260618 22:31:44.602341 29904 consensus_queue.cc:260] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bb454d1d9144588f883147d7ec1d2c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35055 } } peers { permanent_uuid: "aa4b7fe00f66474d9440aca38859b163" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40053 } } peers { permanent_uuid: "7381d19ef72743d6959c83e501f6be2b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36659 } }
I20260618 22:31:44.602563 29903 consensus_queue.cc:260] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bb454d1d9144588f883147d7ec1d2c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35055 } } peers { permanent_uuid: "aa4b7fe00f66474d9440aca38859b163" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40053 } } peers { permanent_uuid: "7381d19ef72743d6959c83e501f6be2b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36659 } }
I20260618 22:31:44.608189 29903 ts_tablet_manager.cc:1428] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c: Time spent starting tablet: real 0.017s user 0.014s sys 0.004s
I20260618 22:31:44.608388 29911 raft_consensus.cc:482] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:31:44.608481 29884 heartbeater.cc:503] Master 127.10.242.254:38185 was elected leader, sending a full tablet report...
I20260618 22:31:44.608937 29911 raft_consensus.cc:504] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bb454d1d9144588f883147d7ec1d2c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35055 } } peers { permanent_uuid: "aa4b7fe00f66474d9440aca38859b163" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40053 } } peers { permanent_uuid: "7381d19ef72743d6959c83e501f6be2b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36659 } }
I20260618 22:31:44.610941 29904 ts_tablet_manager.cc:1428] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163: Time spent starting tablet: real 0.020s user 0.014s sys 0.007s
I20260618 22:31:44.611335 29911 leader_election.cc:288] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2bb454d1d9144588f883147d7ec1d2c (127.10.242.193:35055), 7381d19ef72743d6959c83e501f6be2b (127.10.242.194:36659)
I20260618 22:31:44.618026 29886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:44.611520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:35055 (local address 127.10.242.195:44079)
0618 22:31:44.611869 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:44.611888 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:44.612015 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:44.612472 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:44.612478 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:44.612506 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:44.612730 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:44.612742 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.614010 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.614014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:44.615538 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:44.615551 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.617316 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.617322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:44.617485 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:44.617517 (+ 32us) client_negotiation.cc:769] Sending connection context
0618 22:31:44.617626 (+ 109us) client_negotiation.cc:240] Negotiation successful
0618 22:31:44.617763 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214}
I20260618 22:31:44.618047 29898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:44.611546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:44079 (local address 127.10.242.193:35055)
0618 22:31:44.611733 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:44.611739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:44.611972 (+ 233us) server_negotiation.cc:407] Connection header received
0618 22:31:44.612157 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:44.612163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:44.612232 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:44.612314 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:44.612894 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.613883 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.615681 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.616662 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.616820 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:44.616914 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:44.617659 (+ 745us) server_negotiation.cc:299] Negotiation successful
0618 22:31:44.617804 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:31:44.619132 29901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:44.612076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:47681 (local address 127.10.242.194:36659)
0618 22:31:44.612269 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:44.612274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:44.612962 (+ 688us) server_negotiation.cc:407] Connection header received
0618 22:31:44.613215 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:44.613221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:44.613294 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:44.613388 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:44.613950 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.614933 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.617060 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.618234 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.618416 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:44.618490 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:44.618756 (+ 266us) server_negotiation.cc:299] Negotiation successful
0618 22:31:44.618885 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:31:44.619231 29912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:44.611729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:36659 (local address 127.10.242.195:47681)
0618 22:31:44.612874 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:44.612890 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:44.613049 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:44.613413 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:44.613418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:44.613443 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:44.613789 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:44.613799 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.615097 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.615103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:44.616910 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:44.616923 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.618446 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.618452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:44.618606 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:44.618630 (+ 24us) client_negotiation.cc:769] Sending connection context
0618 22:31:44.618709 (+ 79us) client_negotiation.cc:240] Negotiation successful
0618 22:31:44.618834 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1024,"mutex_wait_us":384,"thread_start_us":431,"threads_started":1}
I20260618 22:31:44.619374 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea3903239bbc4cb4ab06bd3eac6d1137" candidate_uuid: "aa4b7fe00f66474d9440aca38859b163" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2bb454d1d9144588f883147d7ec1d2c" is_pre_election: true
I20260618 22:31:44.619905 29572 raft_consensus.cc:2454] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa4b7fe00f66474d9440aca38859b163 in term 0.
I20260618 22:31:44.620517 29772 leader_election.cc:302] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa4b7fe00f66474d9440aca38859b163, e2bb454d1d9144588f883147d7ec1d2c; no voters:
I20260618 22:31:44.620889 29911 raft_consensus.cc:2781] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:31:44.621007 29911 raft_consensus.cc:482] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:31:44.621129 29911 raft_consensus.cc:3037] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:44.621224 29706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea3903239bbc4cb4ab06bd3eac6d1137" candidate_uuid: "aa4b7fe00f66474d9440aca38859b163" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7381d19ef72743d6959c83e501f6be2b" is_pre_election: true
I20260618 22:31:44.621697 29706 raft_consensus.cc:2454] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa4b7fe00f66474d9440aca38859b163 in term 0.
I20260618 22:31:44.623504 29911 raft_consensus.cc:504] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bb454d1d9144588f883147d7ec1d2c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35055 } } peers { permanent_uuid: "aa4b7fe00f66474d9440aca38859b163" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40053 } } peers { permanent_uuid: "7381d19ef72743d6959c83e501f6be2b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36659 } }
I20260618 22:31:44.624104 29911 leader_election.cc:288] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [CANDIDATE]: Term 1 election: Requested vote from peers e2bb454d1d9144588f883147d7ec1d2c (127.10.242.193:35055), 7381d19ef72743d6959c83e501f6be2b (127.10.242.194:36659)
I20260618 22:31:44.624614 29706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea3903239bbc4cb4ab06bd3eac6d1137" candidate_uuid: "aa4b7fe00f66474d9440aca38859b163" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7381d19ef72743d6959c83e501f6be2b"
I20260618 22:31:44.624606 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea3903239bbc4cb4ab06bd3eac6d1137" candidate_uuid: "aa4b7fe00f66474d9440aca38859b163" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2bb454d1d9144588f883147d7ec1d2c"
I20260618 22:31:44.624826 29706 raft_consensus.cc:3037] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:44.624826 29572 raft_consensus.cc:3037] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:44.628067 29572 raft_consensus.cc:2454] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa4b7fe00f66474d9440aca38859b163 in term 1.
I20260618 22:31:44.628077 29706 raft_consensus.cc:2454] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa4b7fe00f66474d9440aca38859b163 in term 1.
I20260618 22:31:44.628576 29772 leader_election.cc:302] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa4b7fe00f66474d9440aca38859b163, e2bb454d1d9144588f883147d7ec1d2c; no voters:
I20260618 22:31:44.628895 29911 raft_consensus.cc:2781] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:31:44.629048 29911 raft_consensus.cc:686] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [term 1 LEADER]: Becoming Leader. State: Replica: aa4b7fe00f66474d9440aca38859b163, State: Running, Role: LEADER
I20260618 22:31:44.629561 29911 consensus_queue.cc:237] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bb454d1d9144588f883147d7ec1d2c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35055 } } peers { permanent_uuid: "aa4b7fe00f66474d9440aca38859b163" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40053 } } peers { permanent_uuid: "7381d19ef72743d6959c83e501f6be2b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36659 } }
I20260618 22:31:44.634225 29428 catalog_manager.cc:5495] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa4b7fe00f66474d9440aca38859b163 (127.10.242.195). New cstate: current_term: 1 leader_uuid: "aa4b7fe00f66474d9440aca38859b163" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bb454d1d9144588f883147d7ec1d2c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa4b7fe00f66474d9440aca38859b163" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 40053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7381d19ef72743d6959c83e501f6be2b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36659 } health_report { overall_health: UNKNOWN } } }
I20260618 22:31:44.665808 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:44.660679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54764 (local address 127.10.242.195:40053)
0618 22:31:44.660856 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:44.660860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:44.660878 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:31:44.661018 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:44.661021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:44.661116 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:44.661252 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:31:44.661637 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.662329 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.663833 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.664221 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.664319 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:44.664771 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:31:44.665285 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:31:44.665410 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:44.665553 (+ 143us) server_negotiation.cc:299] Negotiation successful
0618 22:31:44.665592 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:31:44.674145 29572 raft_consensus.cc:1261] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c [term 1 FOLLOWER]: Refusing update from remote peer aa4b7fe00f66474d9440aca38859b163: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:31:44.674145 29706 raft_consensus.cc:1261] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b [term 1 FOLLOWER]: Refusing update from remote peer aa4b7fe00f66474d9440aca38859b163: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:31:44.675215 29911 consensus_queue.cc:1035] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7381d19ef72743d6959c83e501f6be2b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:31:44.675521 29909 consensus_queue.cc:1035] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2bb454d1d9144588f883147d7ec1d2c" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 35055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:31:44.686621 29915 mvcc.cc:205] Tried to move back new op lower bound from 7298342521534279680 to 7298342521366462464. Current Snapshot: MvccSnapshot[applied={T|T < 7298342521534279680}]
I20260618 22:31:44.704522 29428 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:31:44.704784 29428 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:31:44.708467 29428 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04e62df9e8b14bff80f1e8db7230ba56: Sending DeleteTablet for 3 replicas of tablet ea3903239bbc4cb4ab06bd3eac6d1137
I20260618 22:31:44.709645 29552 tablet_service.cc:1514] Processing DeleteTablet for tablet ea3903239bbc4cb4ab06bd3eac6d1137 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:31:44 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:41996
I20260618 22:31:44.709690 29819 tablet_service.cc:1514] Processing DeleteTablet for tablet ea3903239bbc4cb4ab06bd3eac6d1137 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:31:44 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:54754
I20260618 22:31:44.710815 29926 tablet_replica.cc:330] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163: stopping tablet replica
I20260618 22:31:44.710919 29686 tablet_service.cc:1514] Processing DeleteTablet for tablet ea3903239bbc4cb4ab06bd3eac6d1137 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:31:44 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:41690
I20260618 22:31:44.711401 29926 raft_consensus.cc:2229] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:31:44.712178 29926 raft_consensus.cc:2258] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:31:44.714473 29926 ts_tablet_manager.cc:1905] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:31:44.715368 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 29489
I20260618 22:31:44.719429 29925 tablet_replica.cc:330] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c: stopping tablet replica
I20260618 22:31:44.720453 29925 raft_consensus.cc:2229] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:31:44.721632 29927 tablet_replica.cc:330] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b: stopping tablet replica
I20260618 22:31:44.722079 29927 raft_consensus.cc:2229] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:31:44.722311 29925 raft_consensus.cc:2258] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:31:44.722594 29927 raft_consensus.cc:2258] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:31:44.724905 29925 ts_tablet_manager.cc:1905] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:31:44.726074 29926 ts_tablet_manager.cc:1918] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:31:44.726300 29926 log.cc:1196] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/ea3903239bbc4cb4ab06bd3eac6d1137
I20260618 22:31:44.726960 29926 ts_tablet_manager.cc:1939] T ea3903239bbc4cb4ab06bd3eac6d1137 P aa4b7fe00f66474d9440aca38859b163: Deleting consensus metadata
I20260618 22:31:44.729182 29415 catalog_manager.cc:4841] TS aa4b7fe00f66474d9440aca38859b163 (127.10.242.195:40053): tablet ea3903239bbc4cb4ab06bd3eac6d1137 (table test-table [id=bf64f28653d94011889472fdf06eb1de]) successfully deleted
I20260618 22:31:44.732484 29925 ts_tablet_manager.cc:1918] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:31:44.732682 29925 log.cc:1196] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/ea3903239bbc4cb4ab06bd3eac6d1137
I20260618 22:31:44.733258 29925 ts_tablet_manager.cc:1939] T ea3903239bbc4cb4ab06bd3eac6d1137 P e2bb454d1d9144588f883147d7ec1d2c: Deleting consensus metadata
I20260618 22:31:44.735494 29927 ts_tablet_manager.cc:1905] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:31:44.735657 29412 catalog_manager.cc:4841] TS e2bb454d1d9144588f883147d7ec1d2c (127.10.242.193:35055): tablet ea3903239bbc4cb4ab06bd3eac6d1137 (table test-table [id=bf64f28653d94011889472fdf06eb1de]) successfully deleted
I20260618 22:31:44.740957 29927 ts_tablet_manager.cc:1918] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:31:44.741164 29927 log.cc:1196] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/ea3903239bbc4cb4ab06bd3eac6d1137
I20260618 22:31:44.741582 29927 ts_tablet_manager.cc:1939] T ea3903239bbc4cb4ab06bd3eac6d1137 P 7381d19ef72743d6959c83e501f6be2b: Deleting consensus metadata
I20260618 22:31:44.743110 29414 catalog_manager.cc:4841] TS 7381d19ef72743d6959c83e501f6be2b (127.10.242.194:36659): tablet ea3903239bbc4cb4ab06bd3eac6d1137 (table test-table [id=bf64f28653d94011889472fdf06eb1de]) successfully deleted
Jun 18 22:31:44 dist-test-slave-0956 krb5kdc[29383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:31:44.750232 29898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:44.716534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42010 (local address 127.10.242.193:35055)
0618 22:31:44.717520 (+ 986us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:44.717527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:44.733452 (+ 15925us) server_negotiation.cc:407] Connection header received
0618 22:31:44.733767 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:44.733773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:44.733860 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:44.733965 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:44.735790 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.736932 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.738958 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.739564 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.739707 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:44.744644 (+ 4937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:44.744659 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:44.744675 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:44.744712 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:44.747508 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:44.748211 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:44.748219 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:44.748222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:44.748348 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:44.748705 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:44.748708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:44.748711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:44.749068 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:44.749397 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:44.749677 (+ 280us) server_negotiation.cc:299] Negotiation successful
0618 22:31:44.749893 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":834}
I20260618 22:31:44.750936 29612 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:44.839597 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 29489
I20260618 22:31:44.856812 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 29623
Jun 18 22:31:44 dist-test-slave-0956 krb5kdc[29383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.194@KRBTEST.COM
I20260618 22:31:44.873524 29901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:44.857563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41704 (local address 127.10.242.194:36659)
0618 22:31:44.857768 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:44.857774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:44.857793 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:31:44.857956 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:44.857962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:44.858046 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:44.858131 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:44.859360 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.860083 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.861373 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:44.861797 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:44.861906 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:44.866387 (+ 4481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:44.866408 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:44.866428 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:44.866477 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:44.870312 (+ 3835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:44.871160 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:44.871171 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:44.871174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:44.871333 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:44.871735 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:44.871740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:44.871744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:44.872218 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:44.872533 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:44.872882 (+ 349us) server_negotiation.cc:299] Negotiation successful
0618 22:31:44.873159 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260618 22:31:44.874239 29746 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:45.002148 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 29623
I20260618 22:31:45.016752 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 29756
Jun 18 22:31:45 dist-test-slave-0956 krb5kdc[29383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.195@KRBTEST.COM
I20260618 22:31:45.035380 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:45.017715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54776 (local address 127.10.242.195:40053)
0618 22:31:45.017890 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:45.017895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:45.017913 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:31:45.018130 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:45.018135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:45.018201 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:45.018298 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:45.020139 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:45.020993 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:45.022839 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:45.023423 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:45.023545 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:45.029032 (+ 5487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:45.029056 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:45.029072 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:45.029246 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:45.032223 (+ 2977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:45.033142 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:45.033150 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:45.033152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:45.033340 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:45.033752 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:45.033756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:45.033758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:45.034146 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:45.034368 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:45.034811 (+ 443us) server_negotiation.cc:299] Negotiation successful
0618 22:31:45.035000 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260618 22:31:45.036278 29879 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:45.158658 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 29756
I20260618 22:31:45.173950 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 29396
I20260618 22:31:45.175016 29458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:45.284305 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 29396
2026-06-18T22:31:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4101 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-18T22:31:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:31:45Z Disabled control of system clock
I20260618 22:31:45.352504 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:39985
--webserver_interface=127.10.242.254
--webserver_port=0
--builtin_ntp_servers=127.10.242.212:40415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:39985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:31:45.674754 29936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:45.675046 29936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:45.675134 29936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:45.682447 29936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:31:45.682574 29936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:45.682610 29936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:45.682665 29936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:31:45.682727 29936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:31:45.692039 29936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:39985
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:39985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:45.693446 29936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:45.695233 29936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:45.706962 29941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:45.707084 29942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:45.707470 29944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:45.707854 29936 server_base.cc:1034] running on GCE node
I20260618 22:31:45.708626 29936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:45.710145 29936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:45.711387 29936 hybrid_clock.cc:648] HybridClock initialized: now 1781821905711313 us; error 56 us; skew 500 ppm
I20260618 22:31:45.711817 29936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:31:45.714314 29936 webserver.cc:460] Webserver started at http://127.10.242.254:37677/ using document root <none> and password file <none>
I20260618 22:31:45.714975 29936 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:45.715071 29936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:45.715332 29936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:45.718393 29936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "f870066f7eef466fa5e05e7bd356cec6"
format_stamp: "Formatted at 2026-06-18 22:31:45 on dist-test-slave-0956"
server_key: "602a39acbf133bb02a4a8416e51f8bc1"
server_key_iv: "4aeacbeddcc811333be62878733cd17d"
server_key_version: "encryptionkey@0"
I20260618 22:31:45.719198 29936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "f870066f7eef466fa5e05e7bd356cec6"
format_stamp: "Formatted at 2026-06-18 22:31:45 on dist-test-slave-0956"
server_key: "602a39acbf133bb02a4a8416e51f8bc1"
server_key_iv: "4aeacbeddcc811333be62878733cd17d"
server_key_version: "encryptionkey@0"
I20260618 22:31:45.724615 29936 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260618 22:31:45.728232 29950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:45.729661 29936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:31:45.729812 29936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "f870066f7eef466fa5e05e7bd356cec6"
format_stamp: "Formatted at 2026-06-18 22:31:45 on dist-test-slave-0956"
server_key: "602a39acbf133bb02a4a8416e51f8bc1"
server_key_iv: "4aeacbeddcc811333be62878733cd17d"
server_key_version: "encryptionkey@0"
I20260618 22:31:45.730007 29936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:45.752446 29936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:45.753309 29936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:45.753552 29936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:45.770974 29936 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:39985
I20260618 22:31:45.770999 30001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:39985 every 8 connection(s)
I20260618 22:31:45.772333 29936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:31:45.774657 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 29936
I20260618 22:31:45.774824 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:31:45.775359 11211 external_mini_cluster.cc:1428] Setting key 4a0013869539119a0060ae3ccf35a1eb
I20260618 22:31:45.776710 30002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:45.789863 30005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:45.777431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57212 (local address 127.10.242.254:39985)
0618 22:31:45.778480 (+ 1049us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:45.778491 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:45.778538 (+ 47us) server_negotiation.cc:407] Connection header received
0618 22:31:45.780177 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:45.780200 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:45.780708 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:45.781177 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:45.781758 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:45.783223 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:45.784867 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:45.785651 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:45.785842 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:45.787126 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:45.787135 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:45.787155 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:45.787836 (+ 681us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:45.787931 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:45.787956 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:45.788205 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:45.788424 (+ 219us) server_negotiation.cc:299] Negotiation successful
0618 22:31:45.788806 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":366,"threads_started":1}
I20260618 22:31:45.790007 30002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6: Bootstrap starting.
I20260618 22:31:45.793650 30002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:45.794613 30002 log.cc:824] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:45.797477 30002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6: No bootstrap required, opened a new log
I20260618 22:31:45.805490 30002 raft_consensus.cc:348] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f870066f7eef466fa5e05e7bd356cec6" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39985 } }
I20260618 22:31:45.805882 30002 raft_consensus.cc:374] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:45.805981 30002 raft_consensus.cc:729] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f870066f7eef466fa5e05e7bd356cec6, State: Initialized, Role: FOLLOWER
I20260618 22:31:45.806587 30002 consensus_queue.cc:260] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f870066f7eef466fa5e05e7bd356cec6" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39985 } }
I20260618 22:31:45.806804 30002 raft_consensus.cc:388] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:31:45.806932 30002 raft_consensus.cc:482] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:31:45.807077 30002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:45.809367 30002 raft_consensus.cc:504] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f870066f7eef466fa5e05e7bd356cec6" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39985 } }
I20260618 22:31:45.809875 30002 leader_election.cc:302] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f870066f7eef466fa5e05e7bd356cec6; no voters:
I20260618 22:31:45.810443 30002 leader_election.cc:288] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:31:45.810671 30007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:31:45.811721 30007 raft_consensus.cc:686] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6 [term 1 LEADER]: Becoming Leader. State: Replica: f870066f7eef466fa5e05e7bd356cec6, State: Running, Role: LEADER
I20260618 22:31:45.812371 30007 consensus_queue.cc:237] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f870066f7eef466fa5e05e7bd356cec6" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39985 } }
I20260618 22:31:45.812865 30002 sys_catalog.cc:564] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:31:45.815199 30008 sys_catalog.cc:455] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f870066f7eef466fa5e05e7bd356cec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f870066f7eef466fa5e05e7bd356cec6" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39985 } } }
I20260618 22:31:45.815687 30008 sys_catalog.cc:458] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:31:45.815275 30009 sys_catalog.cc:455] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f870066f7eef466fa5e05e7bd356cec6. Latest consensus state: current_term: 1 leader_uuid: "f870066f7eef466fa5e05e7bd356cec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f870066f7eef466fa5e05e7bd356cec6" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39985 } } }
I20260618 22:31:45.815975 30009 sys_catalog.cc:458] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:31:45.817950 30016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:31:45.822943 30016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:31:45.831189 30016 catalog_manager.cc:1347] Generated new cluster ID: 9122366eb4eb4628bf9ed183c377cc62
I20260618 22:31:45.831308 30016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:31:45.875933 30016 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:31:45.877360 30016 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:31:45.885385 30016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6: Generated new TSK 0
I20260618 22:31:45.886513 30016 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:31:45.902693 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:39985
--builtin_ntp_servers=127.10.242.212:40415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:31:46.234117 30026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:46.234438 30026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:46.234579 30026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:46.242074 30026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:46.242182 30026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:46.242316 30026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:31:46.251739 30026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:39985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:46.253187 30026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:46.254981 30026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:46.268289 30032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:46.268507 30031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:46.268792 30034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:46.269112 30026 server_base.cc:1034] running on GCE node
I20260618 22:31:46.269711 30026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:46.270661 30026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:46.271870 30026 hybrid_clock.cc:648] HybridClock initialized: now 1781821906271813 us; error 47 us; skew 500 ppm
I20260618 22:31:46.272176 30026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:31:46.279323 30026 webserver.cc:460] Webserver started at http://127.10.242.193:40195/ using document root <none> and password file <none>
I20260618 22:31:46.280052 30026 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:46.280144 30026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:46.280407 30026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:46.283420 30026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "1bd6d9215a4a4a76bac716de479ceea8"
format_stamp: "Formatted at 2026-06-18 22:31:46 on dist-test-slave-0956"
server_key: "5b5e529d1da5376d1765ca3c3f4d02d6"
server_key_iv: "5487fb62b134eebc18773bcbc300244c"
server_key_version: "encryptionkey@0"
I20260618 22:31:46.284220 30026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "1bd6d9215a4a4a76bac716de479ceea8"
format_stamp: "Formatted at 2026-06-18 22:31:46 on dist-test-slave-0956"
server_key: "5b5e529d1da5376d1765ca3c3f4d02d6"
server_key_iv: "5487fb62b134eebc18773bcbc300244c"
server_key_version: "encryptionkey@0"
I20260618 22:31:46.289753 30026 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260618 22:31:46.293637 30040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:46.295265 30026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:31:46.295449 30026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "1bd6d9215a4a4a76bac716de479ceea8"
format_stamp: "Formatted at 2026-06-18 22:31:46 on dist-test-slave-0956"
server_key: "5b5e529d1da5376d1765ca3c3f4d02d6"
server_key_iv: "5487fb62b134eebc18773bcbc300244c"
server_key_version: "encryptionkey@0"
I20260618 22:31:46.295671 30026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:46.317478 30026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:46.318379 30026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:46.318676 30026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:46.319676 30026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:46.321264 30026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:46.321363 30026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:46.321470 30026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:46.321535 30026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:46.350909 30026 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:45177
I20260618 22:31:46.350977 30152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:45177 every 8 connection(s)
I20260618 22:31:46.352226 30026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:31:46.356081 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 30026
I20260618 22:31:46.356252 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:31:46.356750 11211 external_mini_cluster.cc:1428] Setting key 717478b7378f1d473d4fe016156728fc
I20260618 22:31:46.359615 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:39985
--builtin_ntp_servers=127.10.242.212:40415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260618 22:31:46.364938 30155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:46.354764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:50967 (local address 127.10.242.254:39985)
0618 22:31:46.355240 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:46.355245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:46.356429 (+ 1184us) server_negotiation.cc:407] Connection header received
0618 22:31:46.358590 (+ 2161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:46.358596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:46.358675 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:46.358763 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:46.359936 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:46.360684 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:46.362304 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:46.362703 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:46.362813 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:46.364137 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:46.364142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:46.364145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:46.364165 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:46.364202 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:46.364214 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:46.364285 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:46.364492 (+ 207us) server_negotiation.cc:299] Negotiation successful
0618 22:31:46.364595 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":227,"threads_started":1}
I20260618 22:31:46.366183 30156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:46.355304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:39985 (local address 127.10.242.193:50967)
0618 22:31:46.356258 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:46.356307 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:46.358284 (+ 1977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:46.359001 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:46.359009 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:46.359106 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:31:46.359742 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:46.359773 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:46.360830 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:46.360834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:46.362163 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:46.362177 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:46.362918 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:46.362925 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:46.363160 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:46.363805 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:31:46.363807 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:31:46.363850 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:31:46.363939 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:31:46.364296 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:31:46.364308 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:31:46.364316 (+ 8us) client_negotiation.cc:769] Sending connection context
0618 22:31:46.364472 (+ 156us) client_negotiation.cc:240] Negotiation successful
0618 22:31:46.364901 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":292,"threads_started":1}
I20260618 22:31:46.367719 30153 heartbeater.cc:347] Connected to a master server at 127.10.242.254:39985
I20260618 22:31:46.368038 30153 heartbeater.cc:464] Registering TS with master...
I20260618 22:31:46.368909 30153 heartbeater.cc:511] Master 127.10.242.254:39985 requested a full tablet report, sending...
I20260618 22:31:46.371148 29967 ts_manager.cc:194] Registered new tserver with Master: 1bd6d9215a4a4a76bac716de479ceea8 (127.10.242.193:45177)
I20260618 22:31:46.372661 29967 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.193:50967
W20260618 22:31:46.681056 30157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:46.681402 30157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:46.681532 30157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:46.688834 30157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:46.688957 30157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:46.689114 30157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:31:46.698410 30157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:39985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:46.699733 30157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:46.701534 30157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:46.714560 30162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:46.714653 30163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:46.714895 30165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:46.715812 30157 server_base.cc:1034] running on GCE node
I20260618 22:31:46.716477 30157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:46.717554 30157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:46.718803 30157 hybrid_clock.cc:648] HybridClock initialized: now 1781821906718744 us; error 37 us; skew 500 ppm
I20260618 22:31:46.719182 30157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:31:46.721575 30157 webserver.cc:460] Webserver started at http://127.10.242.194:35067/ using document root <none> and password file <none>
I20260618 22:31:46.722280 30157 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:46.722380 30157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:46.722679 30157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:46.725677 30157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "2f6f779e83a2435e9682381af5601c7e"
format_stamp: "Formatted at 2026-06-18 22:31:46 on dist-test-slave-0956"
server_key: "698f4949f14fb51bc5fd59a1e02cee0c"
server_key_iv: "5837bf92968d14366c95b27d690d3bc1"
server_key_version: "encryptionkey@0"
I20260618 22:31:46.726472 30157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "2f6f779e83a2435e9682381af5601c7e"
format_stamp: "Formatted at 2026-06-18 22:31:46 on dist-test-slave-0956"
server_key: "698f4949f14fb51bc5fd59a1e02cee0c"
server_key_iv: "5837bf92968d14366c95b27d690d3bc1"
server_key_version: "encryptionkey@0"
I20260618 22:31:46.731648 30157 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:31:46.735481 30171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:46.736943 30157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:31:46.737138 30157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "2f6f779e83a2435e9682381af5601c7e"
format_stamp: "Formatted at 2026-06-18 22:31:46 on dist-test-slave-0956"
server_key: "698f4949f14fb51bc5fd59a1e02cee0c"
server_key_iv: "5837bf92968d14366c95b27d690d3bc1"
server_key_version: "encryptionkey@0"
I20260618 22:31:46.737360 30157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:46.756942 30157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:46.757946 30157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:46.758275 30157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:46.759306 30157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:46.760757 30157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:46.760849 30157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:46.760955 30157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:46.761022 30157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:46.794296 30157 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:44449
I20260618 22:31:46.794353 30283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:44449 every 8 connection(s)
I20260618 22:31:46.795567 30157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:31:46.804304 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 30157
I20260618 22:31:46.804487 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:31:46.805071 11211 external_mini_cluster.cc:1428] Setting key 43a56363db659f31efd7738bca06c426
I20260618 22:31:46.808132 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:39985
--builtin_ntp_servers=127.10.242.212:40415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260618 22:31:46.808614 30155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:46.798123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:38461 (local address 127.10.242.254:39985)
0618 22:31:46.798352 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:46.798358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:46.799916 (+ 1558us) server_negotiation.cc:407] Connection header received
0618 22:31:46.801708 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:46.801712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:46.801783 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:46.801875 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:46.802919 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:46.803668 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:46.805870 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:46.806317 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:46.806413 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:46.807871 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:46.807876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:46.807879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:46.807897 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:46.807929 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:46.807941 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:46.808007 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:46.808253 (+ 246us) server_negotiation.cc:299] Negotiation successful
0618 22:31:46.808362 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260618 22:31:46.810001 30286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:46.798772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:39985 (local address 127.10.242.194:38461)
0618 22:31:46.799746 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:46.799786 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:46.801438 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:46.802059 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:46.802067 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:46.802172 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:31:46.802764 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:46.802782 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:46.803799 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:46.803803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:46.805725 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:46.805738 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:46.806531 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:46.806538 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:46.806840 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:46.807495 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:31:46.807499 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:31:46.807555 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:31:46.807664 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:31:46.808031 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:31:46.808041 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:31:46.808049 (+ 8us) client_negotiation.cc:769] Sending connection context
0618 22:31:46.808243 (+ 194us) client_negotiation.cc:240] Negotiation successful
0618 22:31:46.808646 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":325,"threads_started":1}
I20260618 22:31:46.811484 30284 heartbeater.cc:347] Connected to a master server at 127.10.242.254:39985
I20260618 22:31:46.811800 30284 heartbeater.cc:464] Registering TS with master...
I20260618 22:31:46.812536 30284 heartbeater.cc:511] Master 127.10.242.254:39985 requested a full tablet report, sending...
I20260618 22:31:46.814049 29967 ts_manager.cc:194] Registered new tserver with Master: 2f6f779e83a2435e9682381af5601c7e (127.10.242.194:44449)
I20260618 22:31:46.815043 29967 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.194:38461
W20260618 22:31:47.131682 30287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:47.131981 30287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:47.132122 30287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:47.139501 30287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:47.139622 30287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:47.139765 30287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:31:47.148974 30287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:39985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:47.150281 30287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:47.151986 30287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:47.164870 30292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:47.164901 30295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:47.164919 30293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:47.166016 30287 server_base.cc:1034] running on GCE node
I20260618 22:31:47.166574 30287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:47.167472 30287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:47.168668 30287 hybrid_clock.cc:648] HybridClock initialized: now 1781821907168622 us; error 37 us; skew 500 ppm
I20260618 22:31:47.168977 30287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:31:47.171212 30287 webserver.cc:460] Webserver started at http://127.10.242.195:38471/ using document root <none> and password file <none>
I20260618 22:31:47.171890 30287 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:47.171979 30287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:47.172279 30287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:47.175230 30287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "6be8541bdd5a4b409ecfd75d0c3b26a3"
format_stamp: "Formatted at 2026-06-18 22:31:47 on dist-test-slave-0956"
server_key: "b28716b2000ba5d1c750afe1b8504212"
server_key_iv: "5a9ba6a2a0441fa7d8432b69e330090d"
server_key_version: "encryptionkey@0"
I20260618 22:31:47.175992 30287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "6be8541bdd5a4b409ecfd75d0c3b26a3"
format_stamp: "Formatted at 2026-06-18 22:31:47 on dist-test-slave-0956"
server_key: "b28716b2000ba5d1c750afe1b8504212"
server_key_iv: "5a9ba6a2a0441fa7d8432b69e330090d"
server_key_version: "encryptionkey@0"
I20260618 22:31:47.180862 30287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260618 22:31:47.184212 30301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:47.185572 30287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:31:47.185729 30287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "6be8541bdd5a4b409ecfd75d0c3b26a3"
format_stamp: "Formatted at 2026-06-18 22:31:47 on dist-test-slave-0956"
server_key: "b28716b2000ba5d1c750afe1b8504212"
server_key_iv: "5a9ba6a2a0441fa7d8432b69e330090d"
server_key_version: "encryptionkey@0"
I20260618 22:31:47.185918 30287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:47.211169 30287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:47.212006 30287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:47.212284 30287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:47.213279 30287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:47.214773 30287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:47.214864 30287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:47.214967 30287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:47.215036 30287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:47.244232 30287 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:36599
I20260618 22:31:47.244299 30413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:36599 every 8 connection(s)
I20260618 22:31:47.245530 30287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:31:47.252818 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 30287
I20260618 22:31:47.252972 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:31:47.253494 11211 external_mini_cluster.cc:1428] Setting key 98ad3c982a218ffbed7a85cb927a6838
I20260618 22:31:47.257884 30155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:47.247909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:39117 (local address 127.10.242.254:39985)
0618 22:31:47.248130 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:47.248137 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:47.249541 (+ 1404us) server_negotiation.cc:407] Connection header received
0618 22:31:47.251215 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:47.251218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:47.251286 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:47.251370 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:47.252363 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.253181 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.255192 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.255610 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.255718 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:47.257073 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:47.257099 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:47.257104 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:47.257125 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:47.257163 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:47.257177 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:47.257250 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:47.257463 (+ 213us) server_negotiation.cc:299] Negotiation successful
0618 22:31:47.257561 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260618 22:31:47.258857 30416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:47.248471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:39985 (local address 127.10.242.195:39117)
0618 22:31:47.249396 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:47.249433 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:47.250995 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:47.251545 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:47.251552 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:47.251664 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:31:47.252213 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:47.252230 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.253319 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.253325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:47.255055 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:47.255072 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.255821 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.255829 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:47.256091 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:47.256767 (+ 676us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:31:47.256770 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:31:47.256815 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:31:47.256897 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:31:47.257262 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:31:47.257273 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:31:47.257280 (+ 7us) client_negotiation.cc:769] Sending connection context
0618 22:31:47.257442 (+ 162us) client_negotiation.cc:240] Negotiation successful
0618 22:31:47.257827 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":249,"threads_started":1}
I20260618 22:31:47.260200 30414 heartbeater.cc:347] Connected to a master server at 127.10.242.254:39985
I20260618 22:31:47.260480 30414 heartbeater.cc:464] Registering TS with master...
I20260618 22:31:47.261193 30414 heartbeater.cc:511] Master 127.10.242.254:39985 requested a full tablet report, sending...
I20260618 22:31:47.262442 29967 ts_manager.cc:194] Registered new tserver with Master: 6be8541bdd5a4b409ecfd75d0c3b26a3 (127.10.242.195:36599)
I20260618 22:31:47.263195 29967 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.195:39117
I20260618 22:31:47.268700 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:31:47.278420 30155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:47.272731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57218 (local address 127.10.242.254:39985)
0618 22:31:47.272909 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:47.272913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:47.273293 (+ 380us) server_negotiation.cc:407] Connection header received
0618 22:31:47.273416 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:47.273420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:47.273509 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:47.273612 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:47.274097 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.274853 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.276329 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.276746 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.276853 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:47.277860 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:47.277865 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:47.277868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:47.277885 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:47.277917 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:47.277929 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:47.278038 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:47.278160 (+ 122us) server_negotiation.cc:299] Negotiation successful
0618 22:31:47.278207 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260618 22:31:47.282465 29967 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:31:47.291870 29967 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:31:47.327955 30426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:47.304266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:44449 (local address 127.0.0.1:60384)
0618 22:31:47.319596 (+ 15330us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:47.319616 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:47.319750 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:47.320822 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:47.320830 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:47.320864 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:47.321490 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:47.321504 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.323307 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.323312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:47.325063 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:47.325100 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.327102 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.327109 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:47.327322 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:47.327381 (+ 59us) client_negotiation.cc:769] Sending connection context
0618 22:31:47.327495 (+ 114us) client_negotiation.cc:240] Negotiation successful
0618 22:31:47.327708 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15199,"thread_start_us":12841,"threads_started":1}
I20260618 22:31:47.328691 30428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:47.304191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:36599 (local address 127.0.0.1:42414)
0618 22:31:47.318094 (+ 13903us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:47.318148 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:47.318377 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:47.321175 (+ 2798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:47.321180 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:47.321206 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:47.321897 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:47.321907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.323707 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.323712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:47.325862 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:47.325879 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.327995 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.328002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:47.328162 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:47.328194 (+ 32us) client_negotiation.cc:769] Sending connection context
0618 22:31:47.328268 (+ 74us) client_negotiation.cc:240] Negotiation successful
0618 22:31:47.328393 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12788,"thread_start_us":15152,"threads_started":1}
I20260618 22:31:47.328809 30427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:47.304337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60384 (local address 127.10.242.194:44449)
0618 22:31:47.315002 (+ 10665us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:47.315012 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:47.319745 (+ 4733us) server_negotiation.cc:407] Connection header received
0618 22:31:47.319970 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:47.319979 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:47.320280 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:47.320488 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:47.321792 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.323148 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.325258 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.326598 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.326743 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:47.326974 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:47.328373 (+ 1399us) server_negotiation.cc:299] Negotiation successful
0618 22:31:47.328505 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10495,"thread_start_us":10384,"threads_started":1}
I20260618 22:31:47.329465 30430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:47.304775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42414 (local address 127.10.242.195:36599)
0618 22:31:47.320246 (+ 15471us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:47.320255 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:47.320290 (+ 35us) server_negotiation.cc:407] Connection header received
0618 22:31:47.320393 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:47.320400 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:47.320667 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:47.320859 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:47.322094 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.323551 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.326068 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.327445 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.327628 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:47.327858 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:47.328979 (+ 1121us) server_negotiation.cc:299] Negotiation successful
0618 22:31:47.329154 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15301,"thread_start_us":15777,"threads_started":1}
I20260618 22:31:47.329638 30429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:47.304588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:45177 (local address 127.0.0.1:45540)
0618 22:31:47.321856 (+ 17268us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:47.321892 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:47.322022 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:47.322848 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:47.322853 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:47.322877 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:47.323277 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:47.323289 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.325158 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.325162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:47.326849 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:47.326863 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.328680 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.328687 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:47.329153 (+ 466us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:47.329185 (+ 32us) client_negotiation.cc:769] Sending connection context
0618 22:31:47.329268 (+ 83us) client_negotiation.cc:240] Negotiation successful
0618 22:31:47.329378 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17135,"thread_start_us":16881,"threads_started":1}
I20260618 22:31:47.330667 30431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:47.304841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45540 (local address 127.10.242.193:45177)
0618 22:31:47.319098 (+ 14257us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:47.319105 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:47.322026 (+ 2921us) server_negotiation.cc:407] Connection header received
0618 22:31:47.322161 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:47.322170 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:47.322422 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:47.322630 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:47.323490 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.324997 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.327068 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.328480 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.328624 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:47.328816 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:47.329320 (+ 504us) server_negotiation.cc:299] Negotiation successful
0618 22:31:47.329485 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14111,"thread_start_us":14238,"threads_started":1}
I20260618 22:31:47.332885 30219 tablet_service.cc:1467] Processing CreateTablet for tablet 75a083453b52406ab3938c867e8b6bb2 (DEFAULT_TABLE table=test-table [id=a3f6a06289454c60af124b3950b57d48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:31:47.332885 30349 tablet_service.cc:1467] Processing CreateTablet for tablet 75a083453b52406ab3938c867e8b6bb2 (DEFAULT_TABLE table=test-table [id=a3f6a06289454c60af124b3950b57d48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:31:47.334493 30219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75a083453b52406ab3938c867e8b6bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:47.334491 30349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75a083453b52406ab3938c867e8b6bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:47.335327 30088 tablet_service.cc:1467] Processing CreateTablet for tablet 75a083453b52406ab3938c867e8b6bb2 (DEFAULT_TABLE table=test-table [id=a3f6a06289454c60af124b3950b57d48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:31:47.336438 30088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75a083453b52406ab3938c867e8b6bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:47.347863 30432 tablet_bootstrap.cc:492] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8: Bootstrap starting.
I20260618 22:31:47.350001 30433 tablet_bootstrap.cc:492] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3: Bootstrap starting.
I20260618 22:31:47.351749 30432 tablet_bootstrap.cc:654] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:47.353284 30432 log.cc:824] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:47.353569 30433 tablet_bootstrap.cc:654] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:47.354919 30433 log.cc:824] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:47.357583 30434 tablet_bootstrap.cc:492] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e: Bootstrap starting.
I20260618 22:31:47.364137 30434 tablet_bootstrap.cc:654] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:47.365381 30434 log.cc:824] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:47.368719 30432 tablet_bootstrap.cc:492] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8: No bootstrap required, opened a new log
I20260618 22:31:47.368922 30434 tablet_bootstrap.cc:492] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e: No bootstrap required, opened a new log
I20260618 22:31:47.368994 30432 ts_tablet_manager.cc:1397] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8: Time spent bootstrapping tablet: real 0.021s user 0.011s sys 0.007s
I20260618 22:31:47.369237 30434 ts_tablet_manager.cc:1397] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260618 22:31:47.369459 30433 tablet_bootstrap.cc:492] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3: No bootstrap required, opened a new log
I20260618 22:31:47.369710 30433 ts_tablet_manager.cc:1397] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.012s
I20260618 22:31:47.376300 30153 heartbeater.cc:503] Master 127.10.242.254:39985 was elected leader, sending a full tablet report...
I20260618 22:31:47.376889 30432 raft_consensus.cc:348] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6f779e83a2435e9682381af5601c7e" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44449 } } peers { permanent_uuid: "6be8541bdd5a4b409ecfd75d0c3b26a3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36599 } } peers { permanent_uuid: "1bd6d9215a4a4a76bac716de479ceea8" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 45177 } }
I20260618 22:31:47.377364 30432 raft_consensus.cc:374] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:47.377465 30432 raft_consensus.cc:729] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bd6d9215a4a4a76bac716de479ceea8, State: Initialized, Role: FOLLOWER
I20260618 22:31:47.378075 30432 consensus_queue.cc:260] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6f779e83a2435e9682381af5601c7e" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44449 } } peers { permanent_uuid: "6be8541bdd5a4b409ecfd75d0c3b26a3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36599 } } peers { permanent_uuid: "1bd6d9215a4a4a76bac716de479ceea8" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 45177 } }
I20260618 22:31:47.379514 30433 raft_consensus.cc:348] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6f779e83a2435e9682381af5601c7e" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44449 } } peers { permanent_uuid: "6be8541bdd5a4b409ecfd75d0c3b26a3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36599 } } peers { permanent_uuid: "1bd6d9215a4a4a76bac716de479ceea8" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 45177 } }
I20260618 22:31:47.379465 30434 raft_consensus.cc:348] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6f779e83a2435e9682381af5601c7e" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44449 } } peers { permanent_uuid: "6be8541bdd5a4b409ecfd75d0c3b26a3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36599 } } peers { permanent_uuid: "1bd6d9215a4a4a76bac716de479ceea8" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 45177 } }
I20260618 22:31:47.379899 30434 raft_consensus.cc:374] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:47.379899 30433 raft_consensus.cc:374] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:47.379985 30433 raft_consensus.cc:729] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6be8541bdd5a4b409ecfd75d0c3b26a3, State: Initialized, Role: FOLLOWER
I20260618 22:31:47.379985 30434 raft_consensus.cc:729] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f6f779e83a2435e9682381af5601c7e, State: Initialized, Role: FOLLOWER
I20260618 22:31:47.380568 30434 consensus_queue.cc:260] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6f779e83a2435e9682381af5601c7e" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44449 } } peers { permanent_uuid: "6be8541bdd5a4b409ecfd75d0c3b26a3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36599 } } peers { permanent_uuid: "1bd6d9215a4a4a76bac716de479ceea8" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 45177 } }
I20260618 22:31:47.380754 30433 consensus_queue.cc:260] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6f779e83a2435e9682381af5601c7e" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44449 } } peers { permanent_uuid: "6be8541bdd5a4b409ecfd75d0c3b26a3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36599 } } peers { permanent_uuid: "1bd6d9215a4a4a76bac716de479ceea8" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 45177 } }
I20260618 22:31:47.384356 30433 ts_tablet_manager.cc:1428] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3: Time spent starting tablet: real 0.015s user 0.011s sys 0.003s
I20260618 22:31:47.384719 30432 ts_tablet_manager.cc:1428] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8: Time spent starting tablet: real 0.016s user 0.010s sys 0.004s
I20260618 22:31:47.384727 30414 heartbeater.cc:503] Master 127.10.242.254:39985 was elected leader, sending a full tablet report...
I20260618 22:31:47.385914 30284 heartbeater.cc:503] Master 127.10.242.254:39985 was elected leader, sending a full tablet report...
I20260618 22:31:47.387151 30434 ts_tablet_manager.cc:1428] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e: Time spent starting tablet: real 0.018s user 0.013s sys 0.004s
W20260618 22:31:47.497215 30415 tablet.cc:2369] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:31:47.504375 30440 raft_consensus.cc:482] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:31:47.504688 30440 raft_consensus.cc:504] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6f779e83a2435e9682381af5601c7e" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44449 } } peers { permanent_uuid: "6be8541bdd5a4b409ecfd75d0c3b26a3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36599 } } peers { permanent_uuid: "1bd6d9215a4a4a76bac716de479ceea8" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 45177 } }
I20260618 22:31:47.506444 30440 leader_election.cc:288] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f6f779e83a2435e9682381af5601c7e (127.10.242.194:44449), 1bd6d9215a4a4a76bac716de479ceea8 (127.10.242.193:45177)
I20260618 22:31:47.511924 30431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:47.506916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:46969 (local address 127.10.242.193:45177)
0618 22:31:47.507091 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:47.507097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:47.507169 (+ 72us) server_negotiation.cc:407] Connection header received
0618 22:31:47.507370 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:47.507374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:47.507446 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:47.507519 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:47.508028 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.508720 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.510346 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.511112 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.511251 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:47.511323 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:47.511536 (+ 213us) server_negotiation.cc:299] Negotiation successful
0618 22:31:47.511668 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:31:47.511914 30416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:47.506886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:45177 (local address 127.10.242.195:46969)
0618 22:31:47.507075 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:47.507093 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:47.507238 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:47.507625 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:47.507631 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:47.507662 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:47.507808 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:47.507816 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.508840 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.508845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:47.510196 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:47.510209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.511252 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.511258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:47.511403 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:47.511426 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:31:47.511501 (+ 75us) client_negotiation.cc:240] Negotiation successful
0618 22:31:47.511645 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260618 22:31:47.512683 30427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:47.506862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:60807 (local address 127.10.242.194:44449)
0618 22:31:47.507074 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:47.507080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:47.507846 (+ 766us) server_negotiation.cc:407] Connection header received
0618 22:31:47.508010 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:47.508014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:47.508080 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:47.508162 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:47.508660 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.509428 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.511170 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.511911 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.512040 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:47.512103 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:47.512392 (+ 289us) server_negotiation.cc:299] Negotiation successful
0618 22:31:47.512508 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:31:47.512890 30108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75a083453b52406ab3938c867e8b6bb2" candidate_uuid: "6be8541bdd5a4b409ecfd75d0c3b26a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bd6d9215a4a4a76bac716de479ceea8" is_pre_election: true
I20260618 22:31:47.513266 30441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:47.506886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:44449 (local address 127.10.242.195:60807)
0618 22:31:47.507744 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:47.507758 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:47.507884 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:47.508193 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:47.508198 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:47.508219 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:47.508509 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:47.508517 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.509567 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.509571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:47.511010 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:47.511022 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.512057 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.512064 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:47.512220 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:47.512242 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:31:47.512837 (+ 595us) client_negotiation.cc:240] Negotiation successful
0618 22:31:47.512957 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"mutex_wait_us":217,"thread_start_us":336,"threads_started":1}
I20260618 22:31:47.513449 30108 raft_consensus.cc:2454] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6be8541bdd5a4b409ecfd75d0c3b26a3 in term 0.
I20260618 22:31:47.514242 30304 leader_election.cc:302] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bd6d9215a4a4a76bac716de479ceea8, 6be8541bdd5a4b409ecfd75d0c3b26a3; no voters:
I20260618 22:31:47.514214 30239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75a083453b52406ab3938c867e8b6bb2" candidate_uuid: "6be8541bdd5a4b409ecfd75d0c3b26a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f6f779e83a2435e9682381af5601c7e" is_pre_election: true
I20260618 22:31:47.514679 30440 raft_consensus.cc:2781] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:31:47.514648 30239 raft_consensus.cc:2454] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6be8541bdd5a4b409ecfd75d0c3b26a3 in term 0.
I20260618 22:31:47.514822 30440 raft_consensus.cc:482] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:31:47.514921 30440 raft_consensus.cc:3037] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:47.517189 30440 raft_consensus.cc:504] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6f779e83a2435e9682381af5601c7e" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44449 } } peers { permanent_uuid: "6be8541bdd5a4b409ecfd75d0c3b26a3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36599 } } peers { permanent_uuid: "1bd6d9215a4a4a76bac716de479ceea8" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 45177 } }
I20260618 22:31:47.517755 30440 leader_election.cc:288] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [CANDIDATE]: Term 1 election: Requested vote from peers 2f6f779e83a2435e9682381af5601c7e (127.10.242.194:44449), 1bd6d9215a4a4a76bac716de479ceea8 (127.10.242.193:45177)
I20260618 22:31:47.518174 30239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75a083453b52406ab3938c867e8b6bb2" candidate_uuid: "6be8541bdd5a4b409ecfd75d0c3b26a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f6f779e83a2435e9682381af5601c7e"
I20260618 22:31:47.518283 30108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75a083453b52406ab3938c867e8b6bb2" candidate_uuid: "6be8541bdd5a4b409ecfd75d0c3b26a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bd6d9215a4a4a76bac716de479ceea8"
I20260618 22:31:47.518383 30239 raft_consensus.cc:3037] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:47.518476 30108 raft_consensus.cc:3037] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:47.521574 30108 raft_consensus.cc:2454] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6be8541bdd5a4b409ecfd75d0c3b26a3 in term 1.
I20260618 22:31:47.521576 30239 raft_consensus.cc:2454] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6be8541bdd5a4b409ecfd75d0c3b26a3 in term 1.
I20260618 22:31:47.521962 30304 leader_election.cc:302] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bd6d9215a4a4a76bac716de479ceea8, 6be8541bdd5a4b409ecfd75d0c3b26a3; no voters:
I20260618 22:31:47.522248 30440 raft_consensus.cc:2781] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:31:47.522679 30440 raft_consensus.cc:686] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [term 1 LEADER]: Becoming Leader. State: Replica: 6be8541bdd5a4b409ecfd75d0c3b26a3, State: Running, Role: LEADER
I20260618 22:31:47.523195 30440 consensus_queue.cc:237] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6f779e83a2435e9682381af5601c7e" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44449 } } peers { permanent_uuid: "6be8541bdd5a4b409ecfd75d0c3b26a3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36599 } } peers { permanent_uuid: "1bd6d9215a4a4a76bac716de479ceea8" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 45177 } }
I20260618 22:31:47.527637 29967 catalog_manager.cc:5495] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6be8541bdd5a4b409ecfd75d0c3b26a3 (127.10.242.195). New cstate: current_term: 1 leader_uuid: "6be8541bdd5a4b409ecfd75d0c3b26a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6f779e83a2435e9682381af5601c7e" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6be8541bdd5a4b409ecfd75d0c3b26a3" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bd6d9215a4a4a76bac716de479ceea8" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 45177 } health_report { overall_health: UNKNOWN } } }
W20260618 22:31:47.547614 30285 tablet.cc:2369] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:31:47.583258 30430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:47.577867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42424 (local address 127.10.242.195:36599)
0618 22:31:47.578031 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:47.578035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:47.578052 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:31:47.578173 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:47.578176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:47.578251 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:47.578372 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:31:47.578713 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.579430 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.580927 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.581406 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.581502 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:47.582149 (+ 647us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:31:47.582693 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:31:47.582835 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:47.582981 (+ 146us) server_negotiation.cc:299] Negotiation successful
0618 22:31:47.583021 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:31:47.591591 30239 raft_consensus.cc:1261] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e [term 1 FOLLOWER]: Refusing update from remote peer 6be8541bdd5a4b409ecfd75d0c3b26a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:31:47.591629 30108 raft_consensus.cc:1261] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8 [term 1 FOLLOWER]: Refusing update from remote peer 6be8541bdd5a4b409ecfd75d0c3b26a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:31:47.592576 30440 consensus_queue.cc:1035] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f6f779e83a2435e9682381af5601c7e" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:31:47.592916 30442 consensus_queue.cc:1035] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bd6d9215a4a4a76bac716de479ceea8" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 45177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 22:31:47.605048 30154 tablet.cc:2369] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:31:47.605471 30445 mvcc.cc:205] Tried to move back new op lower bound from 7298342533484605440 to 7298342533219119104. Current Snapshot: MvccSnapshot[applied={T|T < 7298342533484605440}]
I20260618 22:31:47.605929 30446 mvcc.cc:205] Tried to move back new op lower bound from 7298342533484605440 to 7298342533219119104. Current Snapshot: MvccSnapshot[applied={T|T < 7298342533484605440}]
I20260618 22:31:47.621637 29967 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:31:47.621938 29967 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:31:47.625978 29967 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f870066f7eef466fa5e05e7bd356cec6: Sending DeleteTablet for 3 replicas of tablet 75a083453b52406ab3938c867e8b6bb2
I20260618 22:31:47.627140 30088 tablet_service.cc:1514] Processing DeleteTablet for tablet 75a083453b52406ab3938c867e8b6bb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:31:47 UTC) from {username='slave'} at 127.0.0.1:45540
I20260618 22:31:47.627210 30349 tablet_service.cc:1514] Processing DeleteTablet for tablet 75a083453b52406ab3938c867e8b6bb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:31:47 UTC) from {username='slave'} at 127.0.0.1:42414
I20260618 22:31:47.627911 30219 tablet_service.cc:1514] Processing DeleteTablet for tablet 75a083453b52406ab3938c867e8b6bb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:31:47 UTC) from {username='slave'} at 127.0.0.1:60384
I20260618 22:31:47.628021 30455 tablet_replica.cc:330] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8: stopping tablet replica
I20260618 22:31:47.628422 30455 raft_consensus.cc:2229] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:31:47.628973 30455 raft_consensus.cc:2258] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:31:47.629812 30457 tablet_replica.cc:330] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e: stopping tablet replica
I20260618 22:31:47.630190 30457 raft_consensus.cc:2229] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:31:47.630586 30457 raft_consensus.cc:2258] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:31:47.631318 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 30026
I20260618 22:31:47.634052 30457 ts_tablet_manager.cc:1905] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:31:47.636646 30455 ts_tablet_manager.cc:1905] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:31:47.640789 30457 ts_tablet_manager.cc:1918] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:31:47.640987 30457 log.cc:1196] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/75a083453b52406ab3938c867e8b6bb2
I20260618 22:31:47.641530 30457 ts_tablet_manager.cc:1939] T 75a083453b52406ab3938c867e8b6bb2 P 2f6f779e83a2435e9682381af5601c7e: Deleting consensus metadata
I20260618 22:31:47.642235 30456 tablet_replica.cc:330] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3: stopping tablet replica
I20260618 22:31:47.642757 30456 raft_consensus.cc:2229] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:31:47.643599 29954 catalog_manager.cc:4841] TS 2f6f779e83a2435e9682381af5601c7e (127.10.242.194:44449): tablet 75a083453b52406ab3938c867e8b6bb2 (table test-table [id=a3f6a06289454c60af124b3950b57d48]) successfully deleted
I20260618 22:31:47.643648 30456 raft_consensus.cc:2258] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:31:47.644389 30455 ts_tablet_manager.cc:1918] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:31:47.644559 30455 log.cc:1196] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/75a083453b52406ab3938c867e8b6bb2
I20260618 22:31:47.644991 30455 ts_tablet_manager.cc:1939] T 75a083453b52406ab3938c867e8b6bb2 P 1bd6d9215a4a4a76bac716de479ceea8: Deleting consensus metadata
I20260618 22:31:47.645934 30456 ts_tablet_manager.cc:1905] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:31:47.646528 29953 catalog_manager.cc:4841] TS 1bd6d9215a4a4a76bac716de479ceea8 (127.10.242.193:45177): tablet 75a083453b52406ab3938c867e8b6bb2 (table test-table [id=a3f6a06289454c60af124b3950b57d48]) successfully deleted
I20260618 22:31:47.653745 30456 ts_tablet_manager.cc:1918] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:31:47.653966 30456 log.cc:1196] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/75a083453b52406ab3938c867e8b6bb2
I20260618 22:31:47.654579 30456 ts_tablet_manager.cc:1939] T 75a083453b52406ab3938c867e8b6bb2 P 6be8541bdd5a4b409ecfd75d0c3b26a3: Deleting consensus metadata
I20260618 22:31:47.655541 30431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:47.632755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45552 (local address 127.10.242.193:45177)
0618 22:31:47.632957 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:47.632963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:47.649630 (+ 16667us) server_negotiation.cc:407] Connection header received
0618 22:31:47.649818 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:47.649825 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:47.649914 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:47.650017 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:47.650425 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.651189 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.652487 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.652901 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.653011 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:47.654143 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:47.654155 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:47.654170 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:47.654771 (+ 601us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:47.654822 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:47.654842 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:47.654940 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:47.655108 (+ 168us) server_negotiation.cc:299] Negotiation successful
0618 22:31:47.655223 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260618 22:31:47.656240 30148 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:47.656517 29952 catalog_manager.cc:4841] TS 6be8541bdd5a4b409ecfd75d0c3b26a3 (127.10.242.195:36599): tablet 75a083453b52406ab3938c867e8b6bb2 (table test-table [id=a3f6a06289454c60af124b3950b57d48]) successfully deleted
I20260618 22:31:47.745242 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 30026
I20260618 22:31:47.759665 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 30157
I20260618 22:31:47.766999 30427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:47.760481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60390 (local address 127.10.242.194:44449)
0618 22:31:47.760658 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:47.760663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:47.760681 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:31:47.760819 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:47.760823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:47.760905 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:47.760992 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:47.761499 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.762293 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.764087 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.764493 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.764589 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:47.765685 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:47.765694 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:47.765709 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:47.766306 (+ 597us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:47.766355 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:47.766375 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:47.766464 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:47.766595 (+ 131us) server_negotiation.cc:299] Negotiation successful
0618 22:31:47.766689 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260618 22:31:47.767731 30279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:47.866196 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 30157
I20260618 22:31:47.886062 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 30287
I20260618 22:31:47.893536 30430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:47.886876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42430 (local address 127.10.242.195:36599)
0618 22:31:47.887066 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:47.887072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:47.887194 (+ 122us) server_negotiation.cc:407] Connection header received
0618 22:31:47.887386 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:47.887392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:47.887500 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:47.887589 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:47.888048 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.888828 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.890271 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:47.890791 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:47.890918 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:47.892044 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:47.892054 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:47.892070 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:47.892702 (+ 632us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:47.892764 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:47.892790 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:47.892891 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:47.893042 (+ 151us) server_negotiation.cc:299] Negotiation successful
0618 22:31:47.893197 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:31:47.894189 30409 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:48.014551 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 30287
I20260618 22:31:48.029299 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 29936
I20260618 22:31:48.030333 29997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:48.138880 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 29936
2026-06-18T22:31:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2853 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:31:48 dist-test-slave-0956 krb5kdc[30465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:31:48 dist-test-slave-0956 krb5kdc[30465](info): set up 2 sockets
Jun 18 22:31:48 dist-test-slave-0956 krb5kdc[30465](info): commencing operation
krb5kdc: starting...
W20260618 22:31:49.161021 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:31:49 dist-test-slave-0956 krb5kdc[30465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:31:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:31:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:31:49.400794 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:33979
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:35351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:33979
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:31:49.730883 30478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:49.731213 30478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:49.731346 30478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:49.738754 30478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:31:49.738914 30478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:49.738994 30478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:49.739073 30478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:31:49.739131 30478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:31:49.749617 30478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:35351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:33979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:33979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:49.750969 30478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:49.752254 30478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:49.763505 30483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:49.763550 30484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:49.763769 30486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:49.764607 30478 server_base.cc:1034] running on GCE node
I20260618 22:31:49.765472 30478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:49.766872 30478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:49.768092 30478 hybrid_clock.cc:648] HybridClock initialized: now 1781821909768025 us; error 51 us; skew 500 ppm
I20260618 22:31:49.768524 30478 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:31:49 dist-test-slave-0956 krb5kdc[30465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821909, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:31:49.772468 30478 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:31:49.774343 30478 webserver.cc:460] Webserver started at http://127.10.242.254:35371/ using document root <none> and password file <none>
I20260618 22:31:49.775031 30478 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:49.775136 30478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:49.775429 30478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:49.778458 30478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "ddc9b99ff39e4a79a87d92f59608a72c"
format_stamp: "Formatted at 2026-06-18 22:31:49 on dist-test-slave-0956"
server_key: "0a0a679dc7814997b97a3f4c1701bf6a"
server_key_iv: "0cfabec9a24c700461c79bb1c64186db"
server_key_version: "encryptionkey@0"
I20260618 22:31:49.779201 30478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "ddc9b99ff39e4a79a87d92f59608a72c"
format_stamp: "Formatted at 2026-06-18 22:31:49 on dist-test-slave-0956"
server_key: "0a0a679dc7814997b97a3f4c1701bf6a"
server_key_iv: "0cfabec9a24c700461c79bb1c64186db"
server_key_version: "encryptionkey@0"
I20260618 22:31:49.784492 30478 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260618 22:31:49.788182 30493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:49.789657 30478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:31:49.789822 30478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "ddc9b99ff39e4a79a87d92f59608a72c"
format_stamp: "Formatted at 2026-06-18 22:31:49 on dist-test-slave-0956"
server_key: "0a0a679dc7814997b97a3f4c1701bf6a"
server_key_iv: "0cfabec9a24c700461c79bb1c64186db"
server_key_version: "encryptionkey@0"
I20260618 22:31:49.790009 30478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:49.802772 30478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:49.807040 30478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:49.807312 30478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:49.825126 30478 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:33979
I20260618 22:31:49.825160 30544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:33979 every 8 connection(s)
I20260618 22:31:49.826576 30478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:31:49.830395 30545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:49.834160 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 30478
I20260618 22:31:49.834321 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:31:49.834761 11211 external_mini_cluster.cc:1428] Setting key 20204db7edab63bd935015663d2b9540
I20260618 22:31:49.843495 30545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c: Bootstrap starting.
I20260618 22:31:49.847592 30545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:49.848996 30545 log.cc:824] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:31:49 dist-test-slave-0956 krb5kdc[30465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:31:49.852510 30545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c: No bootstrap required, opened a new log
I20260618 22:31:49.858819 30548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:49.836535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59102 (local address 127.10.242.254:33979)
0618 22:31:49.837335 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:49.837347 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:49.837390 (+ 43us) server_negotiation.cc:407] Connection header received
0618 22:31:49.839072 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:49.839100 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:49.839585 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:49.839967 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:49.841878 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:49.843254 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:49.845205 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:49.845975 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:49.846124 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:49.851340 (+ 5216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:49.851350 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:49.851367 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:49.851404 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:49.854479 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:49.855380 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:49.855387 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:49.855394 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:49.855583 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:49.855960 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:49.855963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:49.855965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:49.856563 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:49.856769 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:49.857327 (+ 558us) server_negotiation.cc:299] Negotiation successful
0618 22:31:49.857748 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":368,"threads_started":1}
I20260618 22:31:49.861807 30545 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddc9b99ff39e4a79a87d92f59608a72c" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33979 } }
I20260618 22:31:49.862239 30545 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:49.862334 30545 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddc9b99ff39e4a79a87d92f59608a72c, State: Initialized, Role: FOLLOWER
I20260618 22:31:49.862870 30545 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddc9b99ff39e4a79a87d92f59608a72c" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33979 } }
I20260618 22:31:49.863094 30545 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:31:49.863219 30545 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:31:49.863374 30545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:49.865664 30545 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddc9b99ff39e4a79a87d92f59608a72c" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33979 } }
I20260618 22:31:49.866174 30545 leader_election.cc:302] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddc9b99ff39e4a79a87d92f59608a72c; no voters:
I20260618 22:31:49.866791 30545 leader_election.cc:288] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:31:49.866909 30550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:31:49.867863 30550 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c [term 1 LEADER]: Becoming Leader. State: Replica: ddc9b99ff39e4a79a87d92f59608a72c, State: Running, Role: LEADER
I20260618 22:31:49.868368 30550 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddc9b99ff39e4a79a87d92f59608a72c" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33979 } }
I20260618 22:31:49.868884 30545 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:31:49.871229 30551 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddc9b99ff39e4a79a87d92f59608a72c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddc9b99ff39e4a79a87d92f59608a72c" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33979 } } }
I20260618 22:31:49.871142 30552 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddc9b99ff39e4a79a87d92f59608a72c. Latest consensus state: current_term: 1 leader_uuid: "ddc9b99ff39e4a79a87d92f59608a72c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddc9b99ff39e4a79a87d92f59608a72c" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33979 } } }
I20260618 22:31:49.871543 30552 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c [sys.catalog]: This master's current role is: LEADER
I20260618 22:31:49.871543 30551 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c [sys.catalog]: This master's current role is: LEADER
I20260618 22:31:49.876484 30557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:31:49.880791 30557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:31:49.889768 30557 catalog_manager.cc:1347] Generated new cluster ID: 444f22697b0a4a0299a936883a90c54b
I20260618 22:31:49.889905 30557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:31:49.930364 30557 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:31:49.931804 30557 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:31:49.943159 30557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c: Generated new TSK 0
I20260618 22:31:49.944387 30557 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
Entry for principal kudu/127.10.242.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab.
I20260618 22:31:50.013477 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:33979
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:35351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:31:50.342253 30571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:50.342542 30571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:50.342638 30571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:50.350036 30571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:50.350164 30571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:50.350314 30571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:31:50.360471 30571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:35351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.193.keytab
--principal=kudu/127.10.242.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:33979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:50.361792 30571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:50.363104 30571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:50.376572 30576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:50.376482 30579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:50.376493 30577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:50.377709 30571 server_base.cc:1034] running on GCE node
I20260618 22:31:50.378343 30571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:50.379283 30571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:50.380494 30571 hybrid_clock.cc:648] HybridClock initialized: now 1781821910380436 us; error 48 us; skew 500 ppm
I20260618 22:31:50.380832 30571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:31:50 dist-test-slave-0956 krb5kdc[30465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821910, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:31:50.384040 30571 init.cc:377] Logged in from keytab as kudu/127.10.242.193@KRBTEST.COM (short username kudu)
I20260618 22:31:50.385541 30571 webserver.cc:460] Webserver started at http://127.10.242.193:40097/ using document root <none> and password file <none>
I20260618 22:31:50.386215 30571 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:50.386333 30571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:50.386634 30571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:50.389638 30571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "e2c103f20f0842899dc981a85dda560b"
format_stamp: "Formatted at 2026-06-18 22:31:50 on dist-test-slave-0956"
server_key: "f496bdffea52bf27639f08a2951ab519"
server_key_iv: "63ae255bf009e24c6b960e770f64b9e3"
server_key_version: "encryptionkey@0"
I20260618 22:31:50.390406 30571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "e2c103f20f0842899dc981a85dda560b"
format_stamp: "Formatted at 2026-06-18 22:31:50 on dist-test-slave-0956"
server_key: "f496bdffea52bf27639f08a2951ab519"
server_key_iv: "63ae255bf009e24c6b960e770f64b9e3"
server_key_version: "encryptionkey@0"
I20260618 22:31:50.395794 30571 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:31:50.399518 30586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:50.400942 30571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:31:50.401142 30571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "e2c103f20f0842899dc981a85dda560b"
format_stamp: "Formatted at 2026-06-18 22:31:50 on dist-test-slave-0956"
server_key: "f496bdffea52bf27639f08a2951ab519"
server_key_iv: "63ae255bf009e24c6b960e770f64b9e3"
server_key_version: "encryptionkey@0"
I20260618 22:31:50.401331 30571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:50.419580 30571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:50.423851 30571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:50.424129 30571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:50.425177 30571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:50.426595 30571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:50.426676 30571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:50.426831 30571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:50.426883 30571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:50.457809 30571 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:38433
I20260618 22:31:50.457857 30698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:38433 every 8 connection(s)
I20260618 22:31:50.459088 30571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:31:50.460807 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 30571
I20260618 22:31:50.460981 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:31:50.461438 11211 external_mini_cluster.cc:1428] Setting key debc97d5c078950d49b52288bf309f33
Jun 18 22:31:50 dist-test-slave-0956 krb5kdc[30465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821910, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.193@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:31:50.480759 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:50.461752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:50109 (local address 127.10.242.254:33979)
0618 22:31:50.462294 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:50.462299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:50.463407 (+ 1108us) server_negotiation.cc:407] Connection header received
0618 22:31:50.465914 (+ 2507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:50.465918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:50.465983 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:50.466073 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:50.468008 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:50.468750 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:50.470958 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:50.471368 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:50.471466 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:50.475384 (+ 3918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:50.475389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:50.475392 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:50.475426 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:50.477850 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:50.478652 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:50.478655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:50.478658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:50.478766 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:50.479174 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:50.479176 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:50.479179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:50.479426 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:50.479529 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:50.480333 (+ 804us) server_negotiation.cc:299] Negotiation successful
0618 22:31:50.480504 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":233,"threads_started":1}
I20260618 22:31:50.482154 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:50.462402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:33979 (local address 127.10.242.193:50109)
0618 22:31:50.463222 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:50.463298 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:50.465604 (+ 2306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:50.466352 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:50.466364 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:50.467084 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:31:50.467847 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:50.467872 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:50.468932 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:50.468937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:50.470827 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:50.470844 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:50.471587 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:50.471593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:50.471859 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:50.472605 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:31:50.472634 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:31:50.475145 (+ 2511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:31:50.477971 (+ 2826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:31:50.477980 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:31:50.477999 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:31:50.478553 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:31:50.478851 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:31:50.478855 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:31:50.478858 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:31:50.479085 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:31:50.479530 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:31:50.479537 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:31:50.479953 (+ 416us) client_negotiation.cc:769] Sending connection context
0618 22:31:50.480260 (+ 307us) client_negotiation.cc:240] Negotiation successful
0618 22:31:50.480771 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":292,"threads_started":1}
I20260618 22:31:50.483771 30699 heartbeater.cc:347] Connected to a master server at 127.10.242.254:33979
I20260618 22:31:50.484081 30699 heartbeater.cc:464] Registering TS with master...
I20260618 22:31:50.484797 30699 heartbeater.cc:511] Master 127.10.242.254:33979 requested a full tablet report, sending...
I20260618 22:31:50.486828 30510 ts_manager.cc:194] Registered new tserver with Master: e2c103f20f0842899dc981a85dda560b (127.10.242.193:38433)
WARNING: no policy specified for kudu/127.10.242.194@KRBTEST.COM; defaulting to no policy
I20260618 22:31:50.488372 30510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.193@KRBTEST.COM'} at 127.10.242.193:50109
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
Entry for principal kudu/127.10.242.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab.
I20260618 22:31:50.516068 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:33979
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:35351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:31:50.847326 30705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:50.847697 30705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:50.847846 30705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:50.858101 30705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:50.858260 30705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:50.858465 30705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:31:50.869494 30705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:35351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.194.keytab
--principal=kudu/127.10.242.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:33979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:50.870898 30705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:50.872198 30705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:50.885164 30710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:50.885128 30713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:50.885070 30711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:50.886014 30705 server_base.cc:1034] running on GCE node
I20260618 22:31:50.886636 30705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:50.887610 30705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:50.888827 30705 hybrid_clock.cc:648] HybridClock initialized: now 1781821910888776 us; error 43 us; skew 500 ppm
I20260618 22:31:50.889192 30705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:31:50 dist-test-slave-0956 krb5kdc[30465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821910, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:31:50.892274 30705 init.cc:377] Logged in from keytab as kudu/127.10.242.194@KRBTEST.COM (short username kudu)
I20260618 22:31:50.893879 30705 webserver.cc:460] Webserver started at http://127.10.242.194:44341/ using document root <none> and password file <none>
I20260618 22:31:50.894578 30705 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:50.894692 30705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:50.894984 30705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:50.897961 30705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "2cc61611ffae4b3c92cabc173bd0823b"
format_stamp: "Formatted at 2026-06-18 22:31:50 on dist-test-slave-0956"
server_key: "d2ab07b8cfb8c531746f01db3825d569"
server_key_iv: "eb6139fc9b1b45a5ac21f80495e27aa0"
server_key_version: "encryptionkey@0"
I20260618 22:31:50.898742 30705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "2cc61611ffae4b3c92cabc173bd0823b"
format_stamp: "Formatted at 2026-06-18 22:31:50 on dist-test-slave-0956"
server_key: "d2ab07b8cfb8c531746f01db3825d569"
server_key_iv: "eb6139fc9b1b45a5ac21f80495e27aa0"
server_key_version: "encryptionkey@0"
I20260618 22:31:50.903896 30705 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:31:50.907527 30720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:50.908891 30705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:31:50.909046 30705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "2cc61611ffae4b3c92cabc173bd0823b"
format_stamp: "Formatted at 2026-06-18 22:31:50 on dist-test-slave-0956"
server_key: "d2ab07b8cfb8c531746f01db3825d569"
server_key_iv: "eb6139fc9b1b45a5ac21f80495e27aa0"
server_key_version: "encryptionkey@0"
I20260618 22:31:50.909262 30705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:50.926687 30705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:50.930984 30705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:50.931257 30705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:50.932235 30705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:50.933703 30705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:50.933781 30705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:50.933897 30705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:50.933948 30705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:50.964705 30705 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:36307
I20260618 22:31:50.964751 30832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:36307 every 8 connection(s)
I20260618 22:31:50.965999 30705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:31:50.969385 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 30705
I20260618 22:31:50.969588 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:31:50.970075 11211 external_mini_cluster.cc:1428] Setting key f8812d92e592ef1b5e452bf1120fff43
Jun 18 22:31:50 dist-test-slave-0956 krb5kdc[30465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821910, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.194@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:31:50.987600 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:50.968634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:34061 (local address 127.10.242.254:33979)
0618 22:31:50.968833 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:50.968838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:50.970230 (+ 1392us) server_negotiation.cc:407] Connection header received
0618 22:31:50.972444 (+ 2214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:50.972450 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:50.972564 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:50.972673 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:50.974345 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:50.975305 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:50.977345 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:50.977893 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:50.978009 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:50.981928 (+ 3919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:50.981932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:50.981936 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:50.981970 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:50.984443 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:50.985313 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:50.985317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:50.985321 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:50.985444 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:50.985879 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:50.985881 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:50.985883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:50.986132 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:50.986277 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:50.987026 (+ 749us) server_negotiation.cc:299] Negotiation successful
0618 22:31:50.987270 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:31:50.988638 30835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:50.969214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:33979 (local address 127.10.242.194:34061)
0618 22:31:50.970052 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:50.970103 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:50.972148 (+ 2045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:50.972876 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:50.972885 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:50.973530 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:31:50.974174 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:50.974193 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:50.975477 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:50.975482 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:50.977206 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:50.977220 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:50.978104 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:50.978110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:50.978373 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:50.979076 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:31:50.979101 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:31:50.981658 (+ 2557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:31:50.984589 (+ 2931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:31:50.984597 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:31:50.984616 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:31:50.985203 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:31:50.985540 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:31:50.985544 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:31:50.985547 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:31:50.985781 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:31:50.986271 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:31:50.986279 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:31:50.986624 (+ 345us) client_negotiation.cc:769] Sending connection context
0618 22:31:50.986885 (+ 261us) client_negotiation.cc:240] Negotiation successful
0618 22:31:50.987445 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":321,"threads_started":1}
I20260618 22:31:50.990360 30833 heartbeater.cc:347] Connected to a master server at 127.10.242.254:33979
I20260618 22:31:50.990655 30833 heartbeater.cc:464] Registering TS with master...
I20260618 22:31:50.991410 30833 heartbeater.cc:511] Master 127.10.242.254:33979 requested a full tablet report, sending...
I20260618 22:31:50.993031 30510 ts_manager.cc:194] Registered new tserver with Master: 2cc61611ffae4b3c92cabc173bd0823b (127.10.242.194:36307)
WARNING: no policy specified for kudu/127.10.242.195@KRBTEST.COM; defaulting to no policy
I20260618 22:31:50.994257 30510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.194@KRBTEST.COM'} at 127.10.242.194:34061
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
Entry for principal kudu/127.10.242.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab.
I20260618 22:31:51.024616 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:33979
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:35351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:31:51.356305 30838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:51.356653 30838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:51.356808 30838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:51.364312 30838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:51.364452 30838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:51.364605 30838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:31:51.374532 30838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:35351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.195.keytab
--principal=kudu/127.10.242.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:33979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:51.375872 30838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:51.377509 30838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:51.390488 30844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:51.390560 30843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:51.390537 30846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:51.391129 30838 server_base.cc:1034] running on GCE node
I20260618 22:31:51.391772 30838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:51.392678 30838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:51.393875 30838 hybrid_clock.cc:648] HybridClock initialized: now 1781821911393824 us; error 45 us; skew 500 ppm
I20260618 22:31:51.394168 30838 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:31:51 dist-test-slave-0956 krb5kdc[30465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821911, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:31:51.403223 30838 init.cc:377] Logged in from keytab as kudu/127.10.242.195@KRBTEST.COM (short username kudu)
I20260618 22:31:51.404767 30838 webserver.cc:460] Webserver started at http://127.10.242.195:42995/ using document root <none> and password file <none>
I20260618 22:31:51.405558 30838 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:51.405681 30838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:51.406041 30838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:51.408982 30838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "9b7454738ee34036aa23f20e553dbf60"
format_stamp: "Formatted at 2026-06-18 22:31:51 on dist-test-slave-0956"
server_key: "b2115410869ff580b3293241a3c952a7"
server_key_iv: "855b3417614aee179571c8db0b278acd"
server_key_version: "encryptionkey@0"
I20260618 22:31:51.409790 30838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "9b7454738ee34036aa23f20e553dbf60"
format_stamp: "Formatted at 2026-06-18 22:31:51 on dist-test-slave-0956"
server_key: "b2115410869ff580b3293241a3c952a7"
server_key_iv: "855b3417614aee179571c8db0b278acd"
server_key_version: "encryptionkey@0"
I20260618 22:31:51.415022 30838 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:31:51.418496 30853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:51.419888 30838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:31:51.420032 30838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "9b7454738ee34036aa23f20e553dbf60"
format_stamp: "Formatted at 2026-06-18 22:31:51 on dist-test-slave-0956"
server_key: "b2115410869ff580b3293241a3c952a7"
server_key_iv: "855b3417614aee179571c8db0b278acd"
server_key_version: "encryptionkey@0"
I20260618 22:31:51.420231 30838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:51.436695 30838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:51.440892 30838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:51.441200 30838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:51.442186 30838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:51.443521 30838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:51.443593 30838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:51.443711 30838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:51.443771 30838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:51.474679 30838 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:41305
I20260618 22:31:51.474720 30965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:41305 every 8 connection(s)
I20260618 22:31:51.475975 30838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:31:51.477053 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 30838
I20260618 22:31:51.477240 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:31:51.477746 11211 external_mini_cluster.cc:1428] Setting key 983b7e3aacb5dfaa9903186b89e3788d
Jun 18 22:31:51 dist-test-slave-0956 krb5kdc[30465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821911, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.195@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:31:51.491509 30699 heartbeater.cc:503] Master 127.10.242.254:33979 was elected leader, sending a full tablet report...
I20260618 22:31:51.495688 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:51.478783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:49829 (local address 127.10.242.254:33979)
0618 22:31:51.479020 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:51.479024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:51.480354 (+ 1330us) server_negotiation.cc:407] Connection header received
0618 22:31:51.482211 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:51.482215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:51.482293 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:51.482376 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:51.483771 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.484505 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.486042 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.486441 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.486528 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:51.489760 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:51.489764 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:51.489767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:51.489799 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:51.492614 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:51.493487 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:51.493491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:51.493493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:51.493614 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:51.494031 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:51.494034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:51.494036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:51.494318 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:51.494443 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:51.495140 (+ 697us) server_negotiation.cc:299] Negotiation successful
0618 22:31:51.495362 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260618 22:31:51.496644 30968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:51.479392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:33979 (local address 127.10.242.195:49829)
0618 22:31:51.480192 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:51.480240 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:51.481977 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:51.482562 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:51.482571 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:51.483081 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:31:51.483630 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:51.483648 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.484625 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.484629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:51.485912 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:51.485924 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.486609 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.486614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:51.486802 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:51.487424 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:31:51.487446 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:31:51.489571 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:31:51.492737 (+ 3166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:31:51.492746 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:31:51.492764 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:31:51.493382 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:31:51.493728 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:31:51.493733 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:31:51.493737 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:31:51.493923 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:31:51.494433 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:31:51.494441 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:31:51.494782 (+ 341us) client_negotiation.cc:769] Sending connection context
0618 22:31:51.495012 (+ 230us) client_negotiation.cc:240] Negotiation successful
0618 22:31:51.495553 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":277,"threads_started":1}
I20260618 22:31:51.498066 30966 heartbeater.cc:347] Connected to a master server at 127.10.242.254:33979
I20260618 22:31:51.498355 30966 heartbeater.cc:464] Registering TS with master...
I20260618 22:31:51.499044 30966 heartbeater.cc:511] Master 127.10.242.254:33979 requested a full tablet report, sending...
I20260618 22:31:51.500437 30510 ts_manager.cc:194] Registered new tserver with Master: 9b7454738ee34036aa23f20e553dbf60 (127.10.242.195:41305)
I20260618 22:31:51.501289 30510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.195@KRBTEST.COM'} at 127.10.242.195:49829
I20260618 22:31:51.505410 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:31:51.521442 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:51.509185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59112 (local address 127.10.242.254:33979)
0618 22:31:51.509364 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:51.509369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:51.509818 (+ 449us) server_negotiation.cc:407] Connection header received
0618 22:31:51.509953 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:51.509957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:51.510014 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:51.510089 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:51.511475 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.512177 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.513546 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.513960 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.514049 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:51.516648 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:51.516652 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:51.516656 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:51.516708 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:51.519206 (+ 2498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:51.519815 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:51.519819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:51.519821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:51.519956 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:51.520237 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:51.520239 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:51.520242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:51.520543 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:51.520720 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:51.520984 (+ 264us) server_negotiation.cc:299] Negotiation successful
0618 22:31:51.521207 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260618 22:31:51.525414 30510 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:31:51.532428 30510 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:31:51.571785 30978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:51.544886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:41305 (local address 127.0.0.1:44882)
0618 22:31:51.560774 (+ 15888us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:51.560830 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:51.561052 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:51.564489 (+ 3437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:51.564492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:51.564511 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:51.564935 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:51.564946 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.566982 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.566987 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:51.569376 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:51.569386 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.570671 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.570677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:51.570879 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:51.570938 (+ 59us) client_negotiation.cc:769] Sending connection context
0618 22:31:51.571068 (+ 130us) client_negotiation.cc:240] Negotiation successful
0618 22:31:51.571282 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15556,"mutex_wait_us":97,"thread_start_us":16489,"threads_started":1}
I20260618 22:31:51.572201 30983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:51.545596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42924 (local address 127.10.242.193:38433)
0618 22:31:51.562631 (+ 17035us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:51.562643 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:51.563394 (+ 751us) server_negotiation.cc:407] Connection header received
0618 22:31:51.563522 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:51.563530 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:51.563849 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:51.564062 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:51.564998 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.566502 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.569240 (+ 2738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.570643 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.570813 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:51.570944 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:51.571713 (+ 769us) server_negotiation.cc:299] Negotiation successful
0618 22:31:51.571881 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16849,"thread_start_us":16698,"threads_started":1}
I20260618 22:31:51.572328 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:51.545588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58070 (local address 127.10.242.194:36307)
0618 22:31:51.563410 (+ 17822us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:51.563419 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:51.564374 (+ 955us) server_negotiation.cc:407] Connection header received
0618 22:31:51.564496 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:51.564503 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:51.564705 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:51.564867 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:51.565512 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.566968 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.569008 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.570575 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.570774 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:51.570941 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:51.571885 (+ 944us) server_negotiation.cc:299] Negotiation successful
0618 22:31:51.572042 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17669,"thread_start_us":17547,"threads_started":1}
I20260618 22:31:51.572858 30981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:51.545439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:36307 (local address 127.0.0.1:58070)
0618 22:31:51.564224 (+ 18785us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:51.564242 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:51.564352 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:51.564882 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:51.564886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:51.564917 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:51.565328 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:51.565341 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.567102 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.567107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:51.568861 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:51.568874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.571122 (+ 2248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.571129 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:51.571434 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:51.571523 (+ 89us) client_negotiation.cc:769] Sending connection context
0618 22:31:51.571598 (+ 75us) client_negotiation.cc:240] Negotiation successful
0618 22:31:51.571776 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18656,"mutex_wait_us":169,"thread_start_us":18514,"threads_started":1}
I20260618 22:31:51.572916 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:51.545113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44882 (local address 127.10.242.195:41305)
0618 22:31:51.563886 (+ 18773us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:51.563895 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:51.563927 (+ 32us) server_negotiation.cc:407] Connection header received
0618 22:31:51.564027 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:51.564035 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:51.564278 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:51.564472 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:51.565165 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.566850 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.569517 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.570494 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.570670 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:51.570873 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:51.571113 (+ 240us) server_negotiation.cc:299] Negotiation successful
0618 22:31:51.572621 (+ 1508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18613,"thread_start_us":18468,"threads_started":1}
I20260618 22:31:51.573361 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:51.544945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:38433 (local address 127.0.0.1:42924)
0618 22:31:51.563083 (+ 18138us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:51.563101 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:51.563255 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:51.564251 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:51.564261 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:51.564292 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:51.564765 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:51.564783 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.566628 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.566634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:51.569068 (+ 2434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:51.569106 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.571135 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.571140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:51.571428 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:51.571517 (+ 89us) client_negotiation.cc:769] Sending connection context
0618 22:31:51.571598 (+ 81us) client_negotiation.cc:240] Negotiation successful
0618 22:31:51.572786 (+ 1188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18020,"mutex_wait_us":1157,"thread_start_us":17871,"threads_started":1}
I20260618 22:31:51.576455 30768 tablet_service.cc:1467] Processing CreateTablet for tablet fa82d804cdb84e148056080f79b46b05 (DEFAULT_TABLE table=test-table [id=3fe60eef06af4178a3a3de593eefbc4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:31:51.576490 30634 tablet_service.cc:1467] Processing CreateTablet for tablet fa82d804cdb84e148056080f79b46b05 (DEFAULT_TABLE table=test-table [id=3fe60eef06af4178a3a3de593eefbc4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:31:51.577700 30634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa82d804cdb84e148056080f79b46b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:51.577703 30768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa82d804cdb84e148056080f79b46b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:51.581794 30901 tablet_service.cc:1467] Processing CreateTablet for tablet fa82d804cdb84e148056080f79b46b05 (DEFAULT_TABLE table=test-table [id=3fe60eef06af4178a3a3de593eefbc4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:31:51.583346 30901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa82d804cdb84e148056080f79b46b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:51.589910 30984 tablet_bootstrap.cc:492] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b: Bootstrap starting.
I20260618 22:31:51.593053 30984 tablet_bootstrap.cc:654] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:51.593930 30985 tablet_bootstrap.cc:492] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b: Bootstrap starting.
I20260618 22:31:51.594519 30984 log.cc:824] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:51.597535 30987 tablet_bootstrap.cc:492] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60: Bootstrap starting.
I20260618 22:31:51.597574 30985 tablet_bootstrap.cc:654] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:51.597813 30984 tablet_bootstrap.cc:492] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b: No bootstrap required, opened a new log
I20260618 22:31:51.598158 30984 ts_tablet_manager.cc:1397] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.005s
I20260618 22:31:51.598810 30985 log.cc:824] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:51.600929 30987 tablet_bootstrap.cc:654] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:51.601902 30987 log.cc:824] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:51.602665 30985 tablet_bootstrap.cc:492] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b: No bootstrap required, opened a new log
I20260618 22:31:51.602938 30985 ts_tablet_manager.cc:1397] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260618 22:31:51.604225 30987 tablet_bootstrap.cc:492] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60: No bootstrap required, opened a new log
I20260618 22:31:51.604439 30987 ts_tablet_manager.cc:1397] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260618 22:31:51.608362 30984 raft_consensus.cc:348] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b7454738ee34036aa23f20e553dbf60" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41305 } } peers { permanent_uuid: "e2c103f20f0842899dc981a85dda560b" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 38433 } } peers { permanent_uuid: "2cc61611ffae4b3c92cabc173bd0823b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36307 } }
I20260618 22:31:51.608816 30984 raft_consensus.cc:374] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:51.608959 30984 raft_consensus.cc:729] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc61611ffae4b3c92cabc173bd0823b, State: Initialized, Role: FOLLOWER
I20260618 22:31:51.609624 30984 consensus_queue.cc:260] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b7454738ee34036aa23f20e553dbf60" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41305 } } peers { permanent_uuid: "e2c103f20f0842899dc981a85dda560b" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 38433 } } peers { permanent_uuid: "2cc61611ffae4b3c92cabc173bd0823b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36307 } }
I20260618 22:31:51.610718 30833 heartbeater.cc:503] Master 127.10.242.254:33979 was elected leader, sending a full tablet report...
I20260618 22:31:51.611361 30984 ts_tablet_manager.cc:1428] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260618 22:31:51.613010 30985 raft_consensus.cc:348] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b7454738ee34036aa23f20e553dbf60" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41305 } } peers { permanent_uuid: "e2c103f20f0842899dc981a85dda560b" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 38433 } } peers { permanent_uuid: "2cc61611ffae4b3c92cabc173bd0823b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36307 } }
I20260618 22:31:51.613456 30985 raft_consensus.cc:374] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:51.613545 30985 raft_consensus.cc:729] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c103f20f0842899dc981a85dda560b, State: Initialized, Role: FOLLOWER
I20260618 22:31:51.613835 30987 raft_consensus.cc:348] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b7454738ee34036aa23f20e553dbf60" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41305 } } peers { permanent_uuid: "e2c103f20f0842899dc981a85dda560b" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 38433 } } peers { permanent_uuid: "2cc61611ffae4b3c92cabc173bd0823b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36307 } }
I20260618 22:31:51.614259 30987 raft_consensus.cc:374] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:51.614140 30985 consensus_queue.cc:260] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b7454738ee34036aa23f20e553dbf60" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41305 } } peers { permanent_uuid: "e2c103f20f0842899dc981a85dda560b" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 38433 } } peers { permanent_uuid: "2cc61611ffae4b3c92cabc173bd0823b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36307 } }
I20260618 22:31:51.614367 30987 raft_consensus.cc:729] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b7454738ee34036aa23f20e553dbf60, State: Initialized, Role: FOLLOWER
I20260618 22:31:51.615015 30987 consensus_queue.cc:260] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b7454738ee34036aa23f20e553dbf60" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41305 } } peers { permanent_uuid: "e2c103f20f0842899dc981a85dda560b" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 38433 } } peers { permanent_uuid: "2cc61611ffae4b3c92cabc173bd0823b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36307 } }
I20260618 22:31:51.615767 30985 ts_tablet_manager.cc:1428] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260618 22:31:51.616037 30966 heartbeater.cc:503] Master 127.10.242.254:33979 was elected leader, sending a full tablet report...
I20260618 22:31:51.617841 30987 ts_tablet_manager.cc:1428] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60: Time spent starting tablet: real 0.013s user 0.007s sys 0.003s
W20260618 22:31:51.711426 30700 tablet.cc:2369] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:31:51.717924 30834 tablet.cc:2369] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:31:51.727562 30967 tablet.cc:2369] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:31:51.806897 30991 raft_consensus.cc:482] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:31:51.807238 30991 raft_consensus.cc:504] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b7454738ee34036aa23f20e553dbf60" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41305 } } peers { permanent_uuid: "e2c103f20f0842899dc981a85dda560b" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 38433 } } peers { permanent_uuid: "2cc61611ffae4b3c92cabc173bd0823b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36307 } }
I20260618 22:31:51.809005 30991 leader_election.cc:288] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b7454738ee34036aa23f20e553dbf60 (127.10.242.195:41305), 2cc61611ffae4b3c92cabc173bd0823b (127.10.242.194:36307)
I20260618 22:31:51.815888 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:51.809552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:38811 (local address 127.10.242.194:36307)
0618 22:31:51.809778 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:51.809784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:51.810425 (+ 641us) server_negotiation.cc:407] Connection header received
0618 22:31:51.810600 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:51.810604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:51.810670 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:51.810744 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:51.811390 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.812088 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.814180 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.814937 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.815082 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:51.815162 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:51.815506 (+ 344us) server_negotiation.cc:299] Negotiation successful
0618 22:31:51.815637 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260618 22:31:51.815888 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:51.809415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:42717 (local address 127.10.242.195:41305)
0618 22:31:51.809617 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:51.809623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:51.810334 (+ 711us) server_negotiation.cc:407] Connection header received
0618 22:31:51.810539 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:51.810543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:51.810603 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:51.810696 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:51.811315 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.812050 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.814175 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.814939 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.815089 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:51.815165 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:51.815516 (+ 351us) server_negotiation.cc:299] Negotiation successful
0618 22:31:51.815647 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:31:51.816020 30994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:51.809495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:36307 (local address 127.10.242.193:38811)
0618 22:31:51.810327 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:51.810341 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:51.810453 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:51.810771 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:51.810775 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:51.810801 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:51.811226 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:51.811236 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.812214 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.812219 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:51.814024 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:51.814039 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.815155 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.815162 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:51.815352 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:51.815380 (+ 28us) client_negotiation.cc:769] Sending connection context
0618 22:31:51.815460 (+ 80us) client_negotiation.cc:240] Negotiation successful
0618 22:31:51.815592 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"mutex_wait_us":56,"thread_start_us":450,"threads_started":1}
I20260618 22:31:51.816069 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:51.809424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:41305 (local address 127.10.242.193:42717)
0618 22:31:51.810225 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:51.810241 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:51.810406 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:51.810702 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:51.810707 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:51.810730 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:51.811175 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:51.811189 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.812181 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.812186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:51.814020 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:51.814035 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.815213 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.815219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:51.815377 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:51.815399 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:31:51.815476 (+ 77us) client_negotiation.cc:240] Negotiation successful
0618 22:31:51.815606 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"mutex_wait_us":59,"spinlock_wait_cycles":23424,"thread_start_us":422,"threads_started":1}
I20260618 22:31:51.817139 30788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa82d804cdb84e148056080f79b46b05" candidate_uuid: "e2c103f20f0842899dc981a85dda560b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc61611ffae4b3c92cabc173bd0823b" is_pre_election: true
I20260618 22:31:51.817379 30921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa82d804cdb84e148056080f79b46b05" candidate_uuid: "e2c103f20f0842899dc981a85dda560b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b7454738ee34036aa23f20e553dbf60" is_pre_election: true
I20260618 22:31:51.817656 30788 raft_consensus.cc:2454] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c103f20f0842899dc981a85dda560b in term 0.
I20260618 22:31:51.817850 30921 raft_consensus.cc:2454] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c103f20f0842899dc981a85dda560b in term 0.
I20260618 22:31:51.818324 30589 leader_election.cc:302] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc61611ffae4b3c92cabc173bd0823b, e2c103f20f0842899dc981a85dda560b; no voters:
I20260618 22:31:51.818753 30991 raft_consensus.cc:2781] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:31:51.818884 30991 raft_consensus.cc:482] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:31:51.818998 30991 raft_consensus.cc:3037] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:51.821228 30991 raft_consensus.cc:504] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b7454738ee34036aa23f20e553dbf60" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41305 } } peers { permanent_uuid: "e2c103f20f0842899dc981a85dda560b" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 38433 } } peers { permanent_uuid: "2cc61611ffae4b3c92cabc173bd0823b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36307 } }
I20260618 22:31:51.821894 30991 leader_election.cc:288] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [CANDIDATE]: Term 1 election: Requested vote from peers 9b7454738ee34036aa23f20e553dbf60 (127.10.242.195:41305), 2cc61611ffae4b3c92cabc173bd0823b (127.10.242.194:36307)
I20260618 22:31:51.822428 30921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa82d804cdb84e148056080f79b46b05" candidate_uuid: "e2c103f20f0842899dc981a85dda560b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b7454738ee34036aa23f20e553dbf60"
I20260618 22:31:51.822492 30788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa82d804cdb84e148056080f79b46b05" candidate_uuid: "e2c103f20f0842899dc981a85dda560b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc61611ffae4b3c92cabc173bd0823b"
I20260618 22:31:51.822618 30921 raft_consensus.cc:3037] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:51.822715 30788 raft_consensus.cc:3037] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:51.825133 30788 raft_consensus.cc:2454] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c103f20f0842899dc981a85dda560b in term 1.
I20260618 22:31:51.825290 30921 raft_consensus.cc:2454] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c103f20f0842899dc981a85dda560b in term 1.
I20260618 22:31:51.825660 30589 leader_election.cc:302] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc61611ffae4b3c92cabc173bd0823b, e2c103f20f0842899dc981a85dda560b; no voters:
I20260618 22:31:51.825955 30991 raft_consensus.cc:2781] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:31:51.826397 30991 raft_consensus.cc:686] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [term 1 LEADER]: Becoming Leader. State: Replica: e2c103f20f0842899dc981a85dda560b, State: Running, Role: LEADER
I20260618 22:31:51.826917 30991 consensus_queue.cc:237] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b7454738ee34036aa23f20e553dbf60" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41305 } } peers { permanent_uuid: "e2c103f20f0842899dc981a85dda560b" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 38433 } } peers { permanent_uuid: "2cc61611ffae4b3c92cabc173bd0823b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36307 } }
I20260618 22:31:51.832058 30508 catalog_manager.cc:5495] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b reported cstate change: term changed from 0 to 1, leader changed from <none> to e2c103f20f0842899dc981a85dda560b (127.10.242.193). New cstate: current_term: 1 leader_uuid: "e2c103f20f0842899dc981a85dda560b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b7454738ee34036aa23f20e553dbf60" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2c103f20f0842899dc981a85dda560b" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 38433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cc61611ffae4b3c92cabc173bd0823b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36307 } health_report { overall_health: UNKNOWN } } }
I20260618 22:31:51.855329 30983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:51.850228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42936 (local address 127.10.242.193:38433)
0618 22:31:51.850409 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:51.850413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:51.850429 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:31:51.850515 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:51.850519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:51.850593 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:51.850711 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:31:51.851105 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.851860 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.853218 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.853622 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.853719 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:51.854168 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:31:51.854737 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:31:51.854857 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:51.854946 (+ 89us) server_negotiation.cc:299] Negotiation successful
0618 22:31:51.855017 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260618 22:31:51.863631 30788 raft_consensus.cc:1261] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b [term 1 FOLLOWER]: Refusing update from remote peer e2c103f20f0842899dc981a85dda560b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:31:51.863631 30921 raft_consensus.cc:1261] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60 [term 1 FOLLOWER]: Refusing update from remote peer e2c103f20f0842899dc981a85dda560b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:31:51.864575 30995 consensus_queue.cc:1035] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b7454738ee34036aa23f20e553dbf60" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 41305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:31:51.864926 30991 consensus_queue.cc:1035] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc61611ffae4b3c92cabc173bd0823b" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 36307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:31:51.878126 31000 mvcc.cc:205] Tried to move back new op lower bound from 7298342550983184384 to 7298342550847082496. Current Snapshot: MvccSnapshot[applied={T|T < 7298342550983184384}]
I20260618 22:31:51.879518 30998 mvcc.cc:205] Tried to move back new op lower bound from 7298342550983184384 to 7298342550847082496. Current Snapshot: MvccSnapshot[applied={T|T < 7298342550983184384}]
I20260618 22:31:51.887041 30999 mvcc.cc:205] Tried to move back new op lower bound from 7298342550983184384 to 7298342550847082496. Current Snapshot: MvccSnapshot[applied={T|T < 7298342550983184384}]
I20260618 22:31:51.895368 30509 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:31:51.895651 30509 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:31:51.899329 30509 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ddc9b99ff39e4a79a87d92f59608a72c: Sending DeleteTablet for 3 replicas of tablet fa82d804cdb84e148056080f79b46b05
I20260618 22:31:51.900547 30901 tablet_service.cc:1514] Processing DeleteTablet for tablet fa82d804cdb84e148056080f79b46b05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:31:51 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:44882
I20260618 22:31:51.900583 30634 tablet_service.cc:1514] Processing DeleteTablet for tablet fa82d804cdb84e148056080f79b46b05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:31:51 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:42924
I20260618 22:31:51.900929 30768 tablet_service.cc:1514] Processing DeleteTablet for tablet fa82d804cdb84e148056080f79b46b05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:31:51 UTC) from {username='kudu', principal='kudu/127.10.242.254@KRBTEST.COM'} at 127.0.0.1:58070
I20260618 22:31:51.902854 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 30571
I20260618 22:31:51.912990 31010 tablet_replica.cc:330] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b: stopping tablet replica
I20260618 22:31:51.913573 31010 raft_consensus.cc:2229] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:31:51.914243 31010 raft_consensus.cc:2258] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:31:51.915063 31008 tablet_replica.cc:330] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60: stopping tablet replica
I20260618 22:31:51.915431 31008 raft_consensus.cc:2229] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:31:51.915870 31008 raft_consensus.cc:2258] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:31:51.917974 31008 ts_tablet_manager.cc:1905] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:31:51.918584 31009 tablet_replica.cc:330] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b: stopping tablet replica
I20260618 22:31:51.919092 31009 raft_consensus.cc:2229] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:31:51.919636 31010 ts_tablet_manager.cc:1905] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:31:51.919970 31009 raft_consensus.cc:2258] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:31:51.921816 31009 ts_tablet_manager.cc:1905] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:31:51.925051 31008 ts_tablet_manager.cc:1918] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:31:51.925252 31008 log.cc:1196] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/fa82d804cdb84e148056080f79b46b05
I20260618 22:31:51.925766 31008 ts_tablet_manager.cc:1939] T fa82d804cdb84e148056080f79b46b05 P 9b7454738ee34036aa23f20e553dbf60: Deleting consensus metadata
I20260618 22:31:51.927678 30494 catalog_manager.cc:4841] TS 9b7454738ee34036aa23f20e553dbf60 (127.10.242.195:41305): tablet fa82d804cdb84e148056080f79b46b05 (table test-table [id=3fe60eef06af4178a3a3de593eefbc4d]) successfully deleted
I20260618 22:31:51.928926 31009 ts_tablet_manager.cc:1918] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:31:51.929111 31009 log.cc:1196] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/fa82d804cdb84e148056080f79b46b05
I20260618 22:31:51.929622 31009 ts_tablet_manager.cc:1939] T fa82d804cdb84e148056080f79b46b05 P e2c103f20f0842899dc981a85dda560b: Deleting consensus metadata
I20260618 22:31:51.931209 30495 catalog_manager.cc:4841] TS e2c103f20f0842899dc981a85dda560b (127.10.242.193:38433): tablet fa82d804cdb84e148056080f79b46b05 (table test-table [id=3fe60eef06af4178a3a3de593eefbc4d]) successfully deleted
I20260618 22:31:51.932921 31010 ts_tablet_manager.cc:1918] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:31:51.933127 31010 log.cc:1196] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/fa82d804cdb84e148056080f79b46b05
I20260618 22:31:51.933632 31010 ts_tablet_manager.cc:1939] T fa82d804cdb84e148056080f79b46b05 P 2cc61611ffae4b3c92cabc173bd0823b: Deleting consensus metadata
I20260618 22:31:51.935532 30496 catalog_manager.cc:4841] TS 2cc61611ffae4b3c92cabc173bd0823b (127.10.242.194:36307): tablet fa82d804cdb84e148056080f79b46b05 (table test-table [id=3fe60eef06af4178a3a3de593eefbc4d]) successfully deleted
Jun 18 22:31:51 dist-test-slave-0956 krb5kdc[30465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.193@KRBTEST.COM
I20260618 22:31:51.945421 30983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:51.904235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42948 (local address 127.10.242.193:38433)
0618 22:31:51.904436 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:51.904443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:51.917801 (+ 13358us) server_negotiation.cc:407] Connection header received
0618 22:31:51.918028 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:51.918035 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:51.918131 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:51.918242 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:51.924344 (+ 6102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.925633 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.929203 (+ 3570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:51.929921 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:51.930056 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:51.937825 (+ 7769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:51.937852 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:51.937877 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:51.937930 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:51.941931 (+ 4001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:51.942765 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:51.942776 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:51.942780 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:51.942986 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:51.943464 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:51.943470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:51.943473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:51.943966 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:51.944267 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:51.944736 (+ 469us) server_negotiation.cc:299] Negotiation successful
0618 22:31:51.945007 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260618 22:31:51.947010 30694 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:52.047493 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 30571
I20260618 22:31:52.062804 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 30705
Jun 18 22:31:52 dist-test-slave-0956 krb5kdc[30465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.194@KRBTEST.COM
I20260618 22:31:52.077885 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:52.063720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58078 (local address 127.10.242.194:36307)
0618 22:31:52.063905 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:52.063911 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:52.063930 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:31:52.063991 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:52.063995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:52.064053 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:52.064135 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:52.065417 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:52.066170 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:52.067390 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:52.067774 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:52.067881 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:52.072053 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:52.072069 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:52.072085 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:52.072123 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:52.075164 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:52.075812 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:52.075822 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:52.075826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:52.075951 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:52.076285 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:52.076290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:52.076294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:52.076682 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:52.076967 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:52.077341 (+ 374us) server_negotiation.cc:299] Negotiation successful
0618 22:31:52.077557 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:31:52.078756 30828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:52.165987 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 30705
I20260618 22:31:52.179728 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 30838
Jun 18 22:31:52 dist-test-slave-0956 krb5kdc[30465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.195@KRBTEST.COM
I20260618 22:31:52.195267 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:52.180499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44894 (local address 127.10.242.195:41305)
0618 22:31:52.180720 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:52.180727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:52.180756 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:31:52.180965 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:52.180972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:52.181049 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:52.181177 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:52.182666 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:52.183449 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:52.184771 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:52.185214 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:52.185322 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:52.189694 (+ 4372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:52.189711 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:52.189734 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:52.189769 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:52.192390 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:52.193073 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:52.193187 (+ 114us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:52.193191 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:52.193351 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:52.193712 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:52.193725 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:52.193728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:52.194128 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:52.194394 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:52.194703 (+ 309us) server_negotiation.cc:299] Negotiation successful
0618 22:31:52.194917 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260618 22:31:52.195958 30961 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:52.282372 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 30838
I20260618 22:31:52.297394 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 30478
I20260618 22:31:52.298460 30540 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:52.410218 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 30478
2026-06-18T22:31:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4272 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-18T22:31:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:31:52Z Disabled control of system clock
I20260618 22:31:52.478736 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:46469
--webserver_interface=127.10.242.254
--webserver_port=0
--builtin_ntp_servers=127.10.242.212:36277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:46469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:31:52.798142 31019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:52.798436 31019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:52.798517 31019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:52.805845 31019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:31:52.805968 31019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:52.806005 31019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:52.806064 31019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:31:52.806107 31019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:31:52.815330 31019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:36277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:46469
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:46469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:52.816596 31019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:52.818516 31019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:52.830067 31025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:52.830142 31024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:52.830127 31027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:52.831084 31019 server_base.cc:1034] running on GCE node
I20260618 22:31:52.831959 31019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:52.833523 31019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:52.834754 31019 hybrid_clock.cc:648] HybridClock initialized: now 1781821912834687 us; error 58 us; skew 500 ppm
I20260618 22:31:52.835161 31019 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:31:52.837571 31019 webserver.cc:460] Webserver started at http://127.10.242.254:40671/ using document root <none> and password file <none>
I20260618 22:31:52.838265 31019 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:52.838354 31019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:52.838668 31019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:52.841681 31019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "85e71e4eaf29405697abf29de4ee3676"
format_stamp: "Formatted at 2026-06-18 22:31:52 on dist-test-slave-0956"
server_key: "02d120b12293beb166ab4e958fb64e5e"
server_key_iv: "bc4a170979552c2b79c3c5d9bf1555de"
server_key_version: "encryptionkey@0"
I20260618 22:31:52.842500 31019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "85e71e4eaf29405697abf29de4ee3676"
format_stamp: "Formatted at 2026-06-18 22:31:52 on dist-test-slave-0956"
server_key: "02d120b12293beb166ab4e958fb64e5e"
server_key_iv: "bc4a170979552c2b79c3c5d9bf1555de"
server_key_version: "encryptionkey@0"
I20260618 22:31:52.848664 31019 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260618 22:31:52.852938 31033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:52.854400 31019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:31:52.854568 31019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "85e71e4eaf29405697abf29de4ee3676"
format_stamp: "Formatted at 2026-06-18 22:31:52 on dist-test-slave-0956"
server_key: "02d120b12293beb166ab4e958fb64e5e"
server_key_iv: "bc4a170979552c2b79c3c5d9bf1555de"
server_key_version: "encryptionkey@0"
I20260618 22:31:52.854782 31019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:52.882710 31019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:52.883616 31019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:52.883909 31019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:52.901726 31019 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:46469
I20260618 22:31:52.901748 31084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:46469 every 8 connection(s)
I20260618 22:31:52.903059 31019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:31:52.906860 31085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:52.911413 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 31019
I20260618 22:31:52.911593 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:31:52.912015 11211 external_mini_cluster.cc:1428] Setting key 28fb0a9b08b9949b4c8164bfa59c6474
I20260618 22:31:52.917845 31085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676: Bootstrap starting.
I20260618 22:31:52.920866 31085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:52.921917 31085 log.cc:824] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:52.925076 31085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676: No bootstrap required, opened a new log
I20260618 22:31:52.925119 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:52.913871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49762 (local address 127.10.242.254:46469)
0618 22:31:52.914829 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:52.914841 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:52.914884 (+ 43us) server_negotiation.cc:407] Connection header received
0618 22:31:52.916575 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:52.916598 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:52.917072 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:52.917485 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:52.918019 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:52.919287 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:52.920625 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:52.921442 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:52.921619 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:52.922654 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:52.922666 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:52.922689 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:52.923434 (+ 745us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:52.923531 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:52.923553 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:52.923660 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:52.923860 (+ 200us) server_negotiation.cc:299] Negotiation successful
0618 22:31:52.924173 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":315,"threads_started":1}
I20260618 22:31:52.932817 31085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85e71e4eaf29405697abf29de4ee3676" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 46469 } }
I20260618 22:31:52.933202 31085 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:52.933303 31085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e71e4eaf29405697abf29de4ee3676, State: Initialized, Role: FOLLOWER
I20260618 22:31:52.933838 31085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85e71e4eaf29405697abf29de4ee3676" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 46469 } }
I20260618 22:31:52.934047 31085 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:31:52.934170 31085 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:31:52.934321 31085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:52.936569 31085 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85e71e4eaf29405697abf29de4ee3676" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 46469 } }
I20260618 22:31:52.937064 31085 leader_election.cc:302] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85e71e4eaf29405697abf29de4ee3676; no voters:
I20260618 22:31:52.937638 31085 leader_election.cc:288] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:31:52.937772 31090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:31:52.938380 31090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676 [term 1 LEADER]: Becoming Leader. State: Replica: 85e71e4eaf29405697abf29de4ee3676, State: Running, Role: LEADER
I20260618 22:31:52.938910 31090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85e71e4eaf29405697abf29de4ee3676" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 46469 } }
I20260618 22:31:52.939621 31085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:31:52.941577 31092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85e71e4eaf29405697abf29de4ee3676. Latest consensus state: current_term: 1 leader_uuid: "85e71e4eaf29405697abf29de4ee3676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85e71e4eaf29405697abf29de4ee3676" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 46469 } } }
I20260618 22:31:52.941735 31091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85e71e4eaf29405697abf29de4ee3676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85e71e4eaf29405697abf29de4ee3676" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 46469 } } }
I20260618 22:31:52.942014 31092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676 [sys.catalog]: This master's current role is: LEADER
I20260618 22:31:52.942044 31091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676 [sys.catalog]: This master's current role is: LEADER
I20260618 22:31:52.945272 31098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:31:52.950529 31098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:31:52.959146 31098 catalog_manager.cc:1347] Generated new cluster ID: 957fa194553d485f906e61a38a262df6
I20260618 22:31:52.959271 31098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:31:52.974514 31098 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:31:52.976383 31098 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:31:52.985630 31098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676: Generated new TSK 0
I20260618 22:31:52.986744 31098 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:31:52.991246 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.193:0
--local_ip_for_outbound_sockets=127.10.242.193
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:46469
--builtin_ntp_servers=127.10.242.212:36277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:31:53.320727 31109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:53.321030 31109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:53.321158 31109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:53.328665 31109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:53.328773 31109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:53.328882 31109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.193
I20260618 22:31:53.338245 31109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:36277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.193
--webserver_port=0
--tserver_master_addrs=127.10.242.254:46469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.193
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:53.339517 31109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:53.341363 31109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:53.353856 31117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:53.353921 31115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:53.354239 31114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:53.354908 31109 server_base.cc:1034] running on GCE node
I20260618 22:31:53.355510 31109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:53.356489 31109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:53.357689 31109 hybrid_clock.cc:648] HybridClock initialized: now 1781821913357628 us; error 58 us; skew 500 ppm
I20260618 22:31:53.358055 31109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:31:53.360353 31109 webserver.cc:460] Webserver started at http://127.10.242.193:34739/ using document root <none> and password file <none>
I20260618 22:31:53.361207 31109 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:53.361327 31109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:53.361631 31109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:53.364555 31109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-0/data/instance:
uuid: "661e3b9097b640ddb53c5dffcd3a1ff9"
format_stamp: "Formatted at 2026-06-18 22:31:53 on dist-test-slave-0956"
server_key: "afca06b134a194b927d644ec6c93ee64"
server_key_iv: "5233c6b57917ee2149a47955ecb462f2"
server_key_version: "encryptionkey@0"
I20260618 22:31:53.365420 31109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance:
uuid: "661e3b9097b640ddb53c5dffcd3a1ff9"
format_stamp: "Formatted at 2026-06-18 22:31:53 on dist-test-slave-0956"
server_key: "afca06b134a194b927d644ec6c93ee64"
server_key_iv: "5233c6b57917ee2149a47955ecb462f2"
server_key_version: "encryptionkey@0"
I20260618 22:31:53.370604 31109 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:31:53.374060 31123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:53.375537 31109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:31:53.375685 31109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-0/wal
uuid: "661e3b9097b640ddb53c5dffcd3a1ff9"
format_stamp: "Formatted at 2026-06-18 22:31:53 on dist-test-slave-0956"
server_key: "afca06b134a194b927d644ec6c93ee64"
server_key_iv: "5233c6b57917ee2149a47955ecb462f2"
server_key_version: "encryptionkey@0"
I20260618 22:31:53.375906 31109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:53.408159 31109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:53.409046 31109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:53.409386 31109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:53.410400 31109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:53.411906 31109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:53.412007 31109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:53.412112 31109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:53.412184 31109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:53.443137 31109 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.193:42701
I20260618 22:31:53.443190 31235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.193:42701 every 8 connection(s)
I20260618 22:31:53.444439 31109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-0/data/info.pb
I20260618 22:31:53.454540 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 31109
I20260618 22:31:53.454694 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-0/wal/instance
I20260618 22:31:53.455201 11211 external_mini_cluster.cc:1428] Setting key 85e02c9b1e8bbe930dfc6ec646b9c44e
I20260618 22:31:53.457331 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:53.447043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.193:49347 (local address 127.10.242.254:46469)
0618 22:31:53.447492 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:53.447496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:53.448629 (+ 1133us) server_negotiation.cc:407] Connection header received
0618 22:31:53.450509 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:53.450516 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:53.450612 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:53.450711 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:53.451953 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:53.452678 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:53.454377 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:53.454852 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:53.454951 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:53.456495 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:53.456499 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:53.456503 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:53.456520 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:53.456561 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:53.456573 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:53.456640 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:53.456889 (+ 249us) server_negotiation.cc:299] Negotiation successful
0618 22:31:53.457003 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":258,"threads_started":1}
I20260618 22:31:53.458140 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.194:0
--local_ip_for_outbound_sockets=127.10.242.194
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:46469
--builtin_ntp_servers=127.10.242.212:36277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260618 22:31:53.458545 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:53.447597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:46469 (local address 127.10.242.193:49347)
0618 22:31:53.448477 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:53.448513 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:53.450213 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:53.450989 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:53.450998 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:53.451098 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:31:53.451725 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:53.451752 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:53.452823 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:53.452827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:53.454217 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:53.454234 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:53.455078 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:53.455085 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:53.455393 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:53.456147 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:31:53.456150 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:31:53.456191 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:31:53.456310 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:31:53.456662 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:31:53.456674 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:31:53.456682 (+ 8us) client_negotiation.cc:769] Sending connection context
0618 22:31:53.456879 (+ 197us) client_negotiation.cc:240] Negotiation successful
0618 22:31:53.457343 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":251,"threads_started":1}
I20260618 22:31:53.460035 31236 heartbeater.cc:347] Connected to a master server at 127.10.242.254:46469
I20260618 22:31:53.460336 31236 heartbeater.cc:464] Registering TS with master...
I20260618 22:31:53.461072 31236 heartbeater.cc:511] Master 127.10.242.254:46469 requested a full tablet report, sending...
I20260618 22:31:53.463224 31050 ts_manager.cc:194] Registered new tserver with Master: 661e3b9097b640ddb53c5dffcd3a1ff9 (127.10.242.193:42701)
I20260618 22:31:53.464747 31050 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.193:49347
W20260618 22:31:53.788084 31240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:53.788380 31240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:53.788491 31240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:53.795885 31240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:53.796000 31240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:53.796141 31240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.194
I20260618 22:31:53.805752 31240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:36277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.194
--webserver_port=0
--tserver_master_addrs=127.10.242.254:46469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.194
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:53.807024 31240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:53.808722 31240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:53.821506 31245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:53.821584 31248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:53.821503 31246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:53.822345 31240 server_base.cc:1034] running on GCE node
I20260618 22:31:53.822952 31240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:53.823890 31240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:53.825119 31240 hybrid_clock.cc:648] HybridClock initialized: now 1781821913825040 us; error 52 us; skew 500 ppm
I20260618 22:31:53.825433 31240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:31:53.827836 31240 webserver.cc:460] Webserver started at http://127.10.242.194:41771/ using document root <none> and password file <none>
I20260618 22:31:53.828550 31240 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:53.828646 31240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:53.828945 31240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:53.831862 31240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-1/data/instance:
uuid: "178a828519e5482bb9094a1373f967ae"
format_stamp: "Formatted at 2026-06-18 22:31:53 on dist-test-slave-0956"
server_key: "815b8d3656d4c4f6510f349a72c5e8c9"
server_key_iv: "7651ea2cf4f5aa1f1795107e59a84b39"
server_key_version: "encryptionkey@0"
I20260618 22:31:53.832633 31240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance:
uuid: "178a828519e5482bb9094a1373f967ae"
format_stamp: "Formatted at 2026-06-18 22:31:53 on dist-test-slave-0956"
server_key: "815b8d3656d4c4f6510f349a72c5e8c9"
server_key_iv: "7651ea2cf4f5aa1f1795107e59a84b39"
server_key_version: "encryptionkey@0"
I20260618 22:31:53.837702 31240 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260618 22:31:53.841344 31254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:53.842716 31240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:31:53.842861 31240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-1/wal
uuid: "178a828519e5482bb9094a1373f967ae"
format_stamp: "Formatted at 2026-06-18 22:31:53 on dist-test-slave-0956"
server_key: "815b8d3656d4c4f6510f349a72c5e8c9"
server_key_iv: "7651ea2cf4f5aa1f1795107e59a84b39"
server_key_version: "encryptionkey@0"
I20260618 22:31:53.843081 31240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:53.877424 31240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:53.878330 31240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:53.878620 31240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:53.879664 31240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:53.881240 31240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:53.881346 31240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:53.881451 31240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:53.881521 31240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:53.913103 31240 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.194:34597
I20260618 22:31:53.913162 31366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.194:34597 every 8 connection(s)
I20260618 22:31:53.914433 31240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-1/data/info.pb
I20260618 22:31:53.922652 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 31240
I20260618 22:31:53.922825 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-1/wal/instance
I20260618 22:31:53.923388 11211 external_mini_cluster.cc:1428] Setting key ab71a71c7cfeeedc7b251eb058efc2e3
I20260618 22:31:53.926455 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.195:0
--local_ip_for_outbound_sockets=127.10.242.195
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:46469
--builtin_ntp_servers=127.10.242.212:36277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260618 22:31:53.927686 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:53.917034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.194:40321 (local address 127.10.242.254:46469)
0618 22:31:53.917261 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:53.917268 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:53.918760 (+ 1492us) server_negotiation.cc:407] Connection header received
0618 22:31:53.920537 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:53.920542 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:53.920614 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:53.920705 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:53.921928 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:53.922655 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:53.924721 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:53.925244 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:53.925356 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:53.926786 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:53.926792 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:53.926796 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:53.926828 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:53.926882 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:53.926899 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:53.927007 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:53.927286 (+ 279us) server_negotiation.cc:299] Negotiation successful
0618 22:31:53.927391 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260618 22:31:53.928730 31369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:53.917623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:46469 (local address 127.10.242.194:40321)
0618 22:31:53.918596 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:53.918636 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:53.920249 (+ 1613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:53.920921 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:53.920930 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:53.921024 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:31:53.921750 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:53.921768 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:53.922863 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:53.922869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:53.924580 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:53.924602 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:53.925466 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:53.925475 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:53.925706 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:53.926406 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:31:53.926408 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:31:53.926450 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:31:53.926540 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:31:53.927053 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:31:53.927066 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:31:53.927075 (+ 9us) client_negotiation.cc:769] Sending connection context
0618 22:31:53.927260 (+ 185us) client_negotiation.cc:240] Negotiation successful
0618 22:31:53.927602 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":284,"threads_started":1}
I20260618 22:31:53.930188 31367 heartbeater.cc:347] Connected to a master server at 127.10.242.254:46469
I20260618 22:31:53.930517 31367 heartbeater.cc:464] Registering TS with master...
I20260618 22:31:53.931425 31367 heartbeater.cc:511] Master 127.10.242.254:46469 requested a full tablet report, sending...
I20260618 22:31:53.933063 31050 ts_manager.cc:194] Registered new tserver with Master: 178a828519e5482bb9094a1373f967ae (127.10.242.194:34597)
I20260618 22:31:53.933923 31050 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.194:40321
W20260618 22:31:54.251060 31370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:54.251341 31370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:54.251479 31370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:54.258816 31370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:54.258927 31370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:54.259083 31370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.195
I20260618 22:31:54.268321 31370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:36277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.195
--webserver_port=0
--tserver_master_addrs=127.10.242.254:46469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.195
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:54.269621 31370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:54.271334 31370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:54.283959 31376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:54.283991 31378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:54.284242 31375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:54.284675 31370 server_base.cc:1034] running on GCE node
I20260618 22:31:54.285344 31370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:54.286283 31370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:54.287495 31370 hybrid_clock.cc:648] HybridClock initialized: now 1781821914287449 us; error 31 us; skew 500 ppm
I20260618 22:31:54.287801 31370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:31:54.290077 31370 webserver.cc:460] Webserver started at http://127.10.242.195:35589/ using document root <none> and password file <none>
I20260618 22:31:54.290752 31370 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:54.290853 31370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:54.291150 31370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:54.294060 31370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-2/data/instance:
uuid: "78a6af5f03894a549aca615ced7b723f"
format_stamp: "Formatted at 2026-06-18 22:31:54 on dist-test-slave-0956"
server_key: "cf67919580032edafb1653685d6093c4"
server_key_iv: "f21f1d4faface8af0a97e535196588b7"
server_key_version: "encryptionkey@0"
I20260618 22:31:54.294831 31370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance:
uuid: "78a6af5f03894a549aca615ced7b723f"
format_stamp: "Formatted at 2026-06-18 22:31:54 on dist-test-slave-0956"
server_key: "cf67919580032edafb1653685d6093c4"
server_key_iv: "f21f1d4faface8af0a97e535196588b7"
server_key_version: "encryptionkey@0"
I20260618 22:31:54.300000 31370 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260618 22:31:54.303478 31384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:54.304870 31370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:31:54.305038 31370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-2/wal
uuid: "78a6af5f03894a549aca615ced7b723f"
format_stamp: "Formatted at 2026-06-18 22:31:54 on dist-test-slave-0956"
server_key: "cf67919580032edafb1653685d6093c4"
server_key_iv: "f21f1d4faface8af0a97e535196588b7"
server_key_version: "encryptionkey@0"
I20260618 22:31:54.305299 31370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:54.325910 31370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:54.326767 31370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:54.327054 31370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:54.328078 31370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:31:54.329595 31370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:31:54.329691 31370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:54.329793 31370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:31:54.329859 31370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:54.359891 31370 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.195:44227
I20260618 22:31:54.359952 31496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.195:44227 every 8 connection(s)
I20260618 22:31:54.361188 31370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-2/data/info.pb
I20260618 22:31:54.370662 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 31370
I20260618 22:31:54.370841 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-2/wal/instance
I20260618 22:31:54.371258 11211 external_mini_cluster.cc:1428] Setting key e54dbbbfaa2904f0d13c7942774ab9ee
I20260618 22:31:54.373536 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:54.363629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:40367 (local address 127.10.242.254:46469)
0618 22:31:54.363827 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:54.363831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:54.365215 (+ 1384us) server_negotiation.cc:407] Connection header received
0618 22:31:54.367021 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:54.367026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:54.367115 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:54.367204 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:54.368310 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.369048 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.370605 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.371065 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.371176 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:54.372730 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:54.372736 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:54.372740 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:54.372763 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:54.372802 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:54.372820 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:54.372915 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:54.373194 (+ 279us) server_negotiation.cc:299] Negotiation successful
0618 22:31:54.373284 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260618 22:31:54.374788 31499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:54.364125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:46469 (local address 127.10.242.195:40367)
0618 22:31:54.365028 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:54.365065 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:54.366742 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:54.367471 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:54.367480 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:54.367573 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:31:54.368146 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:54.368163 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.369204 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.369209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:54.370468 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:54.370480 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.371298 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.371306 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:54.371556 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:54.372330 (+ 774us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:31:54.372333 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:31:54.372390 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:31:54.372505 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:31:54.372946 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:31:54.372957 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:31:54.372965 (+ 8us) client_negotiation.cc:769] Sending connection context
0618 22:31:54.373181 (+ 216us) client_negotiation.cc:240] Negotiation successful
0618 22:31:54.373613 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":276,"threads_started":1}
I20260618 22:31:54.375993 31497 heartbeater.cc:347] Connected to a master server at 127.10.242.254:46469
I20260618 22:31:54.376267 31497 heartbeater.cc:464] Registering TS with master...
I20260618 22:31:54.376993 31497 heartbeater.cc:511] Master 127.10.242.254:46469 requested a full tablet report, sending...
I20260618 22:31:54.378275 31050 ts_manager.cc:194] Registered new tserver with Master: 78a6af5f03894a549aca615ced7b723f (127.10.242.195:44227)
I20260618 22:31:54.379051 31050 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.195:40367
I20260618 22:31:54.386428 11211 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:31:54.396180 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:54.390332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49774 (local address 127.10.242.254:46469)
0618 22:31:54.390528 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:54.390535 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:54.390896 (+ 361us) server_negotiation.cc:407] Connection header received
0618 22:31:54.391068 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:54.391071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:54.391135 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:54.391220 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:54.391761 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.392455 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.393849 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.394261 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.394371 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:54.395440 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:54.395446 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:54.395449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:54.395475 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:54.395514 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:54.395528 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:54.395616 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:54.395770 (+ 154us) server_negotiation.cc:299] Negotiation successful
0618 22:31:54.395874 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:31:54.399976 31050 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49774:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:31:54.406847 31050 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:31:54.443198 31509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:54.430331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.195:44227 (local address 127.0.0.1:52560)
0618 22:31:54.435175 (+ 4844us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:54.435194 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:54.435303 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:54.435984 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:54.435988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:54.436011 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:54.436332 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:54.436347 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.438276 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.438281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:54.440448 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:54.440459 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.442186 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.442193 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:54.442431 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:54.442462 (+ 31us) client_negotiation.cc:769] Sending connection context
0618 22:31:54.442723 (+ 261us) client_negotiation.cc:240] Negotiation successful
0618 22:31:54.442850 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4742,"mutex_wait_us":108,"thread_start_us":3810,"threads_started":1}
I20260618 22:31:54.443248 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:54.430622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52560 (local address 127.10.242.195:44227)
0618 22:31:54.434791 (+ 4169us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:54.434800 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:54.435425 (+ 625us) server_negotiation.cc:407] Connection header received
0618 22:31:54.435527 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:54.435534 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:54.435779 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:54.435980 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:54.436551 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.438142 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.440664 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.441668 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.441858 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:54.442059 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:54.442780 (+ 721us) server_negotiation.cc:299] Negotiation successful
0618 22:31:54.442955 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3981,"thread_start_us":3809,"threads_started":1}
I20260618 22:31:54.443436 31512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:54.419698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56766 (local address 127.10.242.194:34597)
0618 22:31:54.434630 (+ 14932us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:54.434648 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:54.434687 (+ 39us) server_negotiation.cc:407] Connection header received
0618 22:31:54.434798 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:54.434807 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:54.435108 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:54.435322 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:54.436084 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.437527 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.440045 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.441592 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.441753 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:54.441996 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:54.442968 (+ 972us) server_negotiation.cc:299] Negotiation successful
0618 22:31:54.443133 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14761,"thread_start_us":15415,"threads_started":1}
I20260618 22:31:54.443899 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:54.418981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:34597 (local address 127.0.0.1:56766)
0618 22:31:54.433908 (+ 14927us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:54.433930 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:54.434057 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:54.435578 (+ 1521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:54.435585 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:54.435621 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:54.435906 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:54.435916 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.437678 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.437683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:54.439839 (+ 2156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:54.439854 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.442144 (+ 2290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.442152 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:54.442400 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:54.442457 (+ 57us) client_negotiation.cc:769] Sending connection context
0618 22:31:54.442593 (+ 136us) client_negotiation.cc:240] Negotiation successful
0618 22:31:54.442798 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14763,"mutex_wait_us":1,"thread_start_us":10089,"threads_started":1}
I20260618 22:31:54.444855 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:54.418843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:42701 (local address 127.0.0.1:50508)
0618 22:31:54.429529 (+ 10686us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:54.429578 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:54.429757 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:54.436223 (+ 6466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:54.436232 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:54.436253 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:54.436630 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:54.436643 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.438569 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.438574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:54.440437 (+ 1863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:54.440452 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.442252 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.442258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:54.444327 (+ 2069us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:54.444363 (+ 36us) client_negotiation.cc:769] Sending connection context
0618 22:31:54.444427 (+ 64us) client_negotiation.cc:240] Negotiation successful
0618 22:31:54.444553 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10373,"mutex_wait_us":40,"thread_start_us":16111,"threads_started":1}
I20260618 22:31:54.446087 31511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:54.419052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50508 (local address 127.10.242.193:42701)
0618 22:31:54.435684 (+ 16632us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:54.435691 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:54.435719 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:31:54.435803 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:54.435810 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:54.436010 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:54.436820 (+ 810us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:54.437109 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.438438 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.440676 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.442055 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.445283 (+ 3228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:54.445514 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:54.445580 (+ 66us) server_negotiation.cc:299] Negotiation successful
0618 22:31:54.445753 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16493,"thread_start_us":16393,"threads_started":1}
I20260618 22:31:54.448311 31432 tablet_service.cc:1467] Processing CreateTablet for tablet e632d857f1d347309f1bcd16217e44cf (DEFAULT_TABLE table=test-table [id=04b38cba17fd4289a73c62826db4d63a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:31:54.449652 31432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e632d857f1d347309f1bcd16217e44cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:54.449169 31302 tablet_service.cc:1467] Processing CreateTablet for tablet e632d857f1d347309f1bcd16217e44cf (DEFAULT_TABLE table=test-table [id=04b38cba17fd4289a73c62826db4d63a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:31:54.450487 31302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e632d857f1d347309f1bcd16217e44cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:54.452466 31171 tablet_service.cc:1467] Processing CreateTablet for tablet e632d857f1d347309f1bcd16217e44cf (DEFAULT_TABLE table=test-table [id=04b38cba17fd4289a73c62826db4d63a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:31:54.453830 31171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e632d857f1d347309f1bcd16217e44cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:54.460546 31515 tablet_bootstrap.cc:492] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f: Bootstrap starting.
I20260618 22:31:54.463626 31515 tablet_bootstrap.cc:654] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:54.465122 31515 log.cc:824] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:54.465824 31516 tablet_bootstrap.cc:492] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae: Bootstrap starting.
I20260618 22:31:54.467794 31236 heartbeater.cc:503] Master 127.10.242.254:46469 was elected leader, sending a full tablet report...
I20260618 22:31:54.468171 31518 tablet_bootstrap.cc:492] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9: Bootstrap starting.
I20260618 22:31:54.468876 31515 tablet_bootstrap.cc:492] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f: No bootstrap required, opened a new log
I20260618 22:31:54.469194 31515 ts_tablet_manager.cc:1397] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260618 22:31:54.469975 31516 tablet_bootstrap.cc:654] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:54.471230 31516 log.cc:824] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:54.472282 31518 tablet_bootstrap.cc:654] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:54.473664 31518 log.cc:824] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:54.476773 31516 tablet_bootstrap.cc:492] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae: No bootstrap required, opened a new log
I20260618 22:31:54.477115 31516 ts_tablet_manager.cc:1397] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260618 22:31:54.478680 31518 tablet_bootstrap.cc:492] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9: No bootstrap required, opened a new log
I20260618 22:31:54.478642 31515 raft_consensus.cc:348] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e3b9097b640ddb53c5dffcd3a1ff9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42701 } } peers { permanent_uuid: "178a828519e5482bb9094a1373f967ae" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34597 } } peers { permanent_uuid: "78a6af5f03894a549aca615ced7b723f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 44227 } }
I20260618 22:31:54.479012 31518 ts_tablet_manager.cc:1397] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260618 22:31:54.479040 31515 raft_consensus.cc:374] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:54.479166 31515 raft_consensus.cc:729] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78a6af5f03894a549aca615ced7b723f, State: Initialized, Role: FOLLOWER
I20260618 22:31:54.479748 31515 consensus_queue.cc:260] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e3b9097b640ddb53c5dffcd3a1ff9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42701 } } peers { permanent_uuid: "178a828519e5482bb9094a1373f967ae" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34597 } } peers { permanent_uuid: "78a6af5f03894a549aca615ced7b723f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 44227 } }
I20260618 22:31:54.481544 31515 ts_tablet_manager.cc:1428] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260618 22:31:54.482574 31497 heartbeater.cc:503] Master 127.10.242.254:46469 was elected leader, sending a full tablet report...
I20260618 22:31:54.482522 31522 raft_consensus.cc:482] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:31:54.482789 31522 raft_consensus.cc:504] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e3b9097b640ddb53c5dffcd3a1ff9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42701 } } peers { permanent_uuid: "178a828519e5482bb9094a1373f967ae" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34597 } } peers { permanent_uuid: "78a6af5f03894a549aca615ced7b723f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 44227 } }
I20260618 22:31:54.487730 31516 raft_consensus.cc:348] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e3b9097b640ddb53c5dffcd3a1ff9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42701 } } peers { permanent_uuid: "178a828519e5482bb9094a1373f967ae" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34597 } } peers { permanent_uuid: "78a6af5f03894a549aca615ced7b723f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 44227 } }
I20260618 22:31:54.488333 31516 raft_consensus.cc:374] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:54.488528 31516 raft_consensus.cc:729] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 178a828519e5482bb9094a1373f967ae, State: Initialized, Role: FOLLOWER
I20260618 22:31:54.489373 31516 consensus_queue.cc:260] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e3b9097b640ddb53c5dffcd3a1ff9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42701 } } peers { permanent_uuid: "178a828519e5482bb9094a1373f967ae" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34597 } } peers { permanent_uuid: "78a6af5f03894a549aca615ced7b723f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 44227 } }
I20260618 22:31:54.489423 31518 raft_consensus.cc:348] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e3b9097b640ddb53c5dffcd3a1ff9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42701 } } peers { permanent_uuid: "178a828519e5482bb9094a1373f967ae" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34597 } } peers { permanent_uuid: "78a6af5f03894a549aca615ced7b723f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 44227 } }
I20260618 22:31:54.489856 31518 raft_consensus.cc:374] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:54.490029 31518 raft_consensus.cc:729] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 661e3b9097b640ddb53c5dffcd3a1ff9, State: Initialized, Role: FOLLOWER
I20260618 22:31:54.490732 31518 consensus_queue.cc:260] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e3b9097b640ddb53c5dffcd3a1ff9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42701 } } peers { permanent_uuid: "178a828519e5482bb9094a1373f967ae" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34597 } } peers { permanent_uuid: "78a6af5f03894a549aca615ced7b723f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 44227 } }
I20260618 22:31:54.491257 31516 ts_tablet_manager.cc:1428] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae: Time spent starting tablet: real 0.014s user 0.010s sys 0.002s
I20260618 22:31:54.491636 31367 heartbeater.cc:503] Master 127.10.242.254:46469 was elected leader, sending a full tablet report...
I20260618 22:31:54.492044 31499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:54.484825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.193:42701 (local address 127.10.242.195:52177)
0618 22:31:54.485255 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:54.485273 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:54.485389 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:54.486471 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:54.486476 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:54.486503 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:54.486720 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:54.486732 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.488161 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.488166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:54.490039 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:54.490057 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.491298 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.491304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:54.491482 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:54.491513 (+ 31us) client_negotiation.cc:769] Sending connection context
0618 22:31:54.491596 (+ 83us) client_negotiation.cc:240] Negotiation successful
0618 22:31:54.491742 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":2}
I20260618 22:31:54.492149 31511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:54.485788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:52177 (local address 127.10.242.193:42701)
0618 22:31:54.485991 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:54.485996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:54.486013 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:31:54.486082 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:54.486087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:54.486182 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:54.486277 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:54.486910 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.487971 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.490246 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.491151 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.491300 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:54.491371 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:54.491717 (+ 346us) server_negotiation.cc:299] Negotiation successful
0618 22:31:54.491879 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:31:54.493310 31191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e632d857f1d347309f1bcd16217e44cf" candidate_uuid: "78a6af5f03894a549aca615ced7b723f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661e3b9097b640ddb53c5dffcd3a1ff9" is_pre_election: true
I20260618 22:31:54.493703 31522 leader_election.cc:288] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 661e3b9097b640ddb53c5dffcd3a1ff9 (127.10.242.193:42701), 178a828519e5482bb9094a1373f967ae (127.10.242.194:34597)
I20260618 22:31:54.493875 31191 raft_consensus.cc:2454] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78a6af5f03894a549aca615ced7b723f in term 0.
I20260618 22:31:54.494769 31385 leader_election.cc:302] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 661e3b9097b640ddb53c5dffcd3a1ff9, 78a6af5f03894a549aca615ced7b723f; no voters:
I20260618 22:31:54.495205 31522 raft_consensus.cc:2781] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:31:54.495342 31522 raft_consensus.cc:482] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:31:54.495432 31522 raft_consensus.cc:3037] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:54.496061 31518 ts_tablet_manager.cc:1428] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
I20260618 22:31:54.498627 31522 raft_consensus.cc:504] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e3b9097b640ddb53c5dffcd3a1ff9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42701 } } peers { permanent_uuid: "178a828519e5482bb9094a1373f967ae" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34597 } } peers { permanent_uuid: "78a6af5f03894a549aca615ced7b723f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 44227 } }
I20260618 22:31:54.499267 31522 leader_election.cc:288] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [CANDIDATE]: Term 1 election: Requested vote from peers 661e3b9097b640ddb53c5dffcd3a1ff9 (127.10.242.193:42701), 178a828519e5482bb9094a1373f967ae (127.10.242.194:34597)
I20260618 22:31:54.499626 31191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e632d857f1d347309f1bcd16217e44cf" candidate_uuid: "78a6af5f03894a549aca615ced7b723f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661e3b9097b640ddb53c5dffcd3a1ff9"
I20260618 22:31:54.499806 31191 raft_consensus.cc:3037] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:54.501672 31499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:54.493462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.194:34597 (local address 127.10.242.195:44571)
0618 22:31:54.494906 (+ 1444us) negotiation.cc:107] Waiting for socket to connect
0618 22:31:54.494925 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:31:54.495085 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:31:54.495595 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:31:54.495599 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:31:54.495626 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:31:54.496830 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:54.496849 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.498186 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.498191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:54.499630 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:31:54.499641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.501052 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.501056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:31:54.501269 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:54.501295 (+ 26us) client_negotiation.cc:769] Sending connection context
0618 22:31:54.501375 (+ 80us) client_negotiation.cc:240] Negotiation successful
0618 22:31:54.501471 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1303}
I20260618 22:31:54.501793 31512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:54.494279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.195:44571 (local address 127.10.242.194:34597)
0618 22:31:54.494452 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:54.494458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:54.494984 (+ 526us) server_negotiation.cc:407] Connection header received
0618 22:31:54.495243 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:54.495248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:54.495345 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:54.495405 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:31:54.496985 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.498040 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.499789 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.500893 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.501042 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:54.501134 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:54.501415 (+ 281us) server_negotiation.cc:299] Negotiation successful
0618 22:31:54.501543 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:31:54.502820 31191 raft_consensus.cc:2454] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78a6af5f03894a549aca615ced7b723f in term 1.
I20260618 22:31:54.502957 31321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e632d857f1d347309f1bcd16217e44cf" candidate_uuid: "78a6af5f03894a549aca615ced7b723f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "178a828519e5482bb9094a1373f967ae"
I20260618 22:31:54.503157 31385 leader_election.cc:302] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 661e3b9097b640ddb53c5dffcd3a1ff9, 78a6af5f03894a549aca615ced7b723f; no voters:
I20260618 22:31:54.502919 31322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e632d857f1d347309f1bcd16217e44cf" candidate_uuid: "78a6af5f03894a549aca615ced7b723f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "178a828519e5482bb9094a1373f967ae" is_pre_election: true
I20260618 22:31:54.503432 31522 raft_consensus.cc:2781] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:31:54.503482 31321 raft_consensus.cc:3037] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:54.503582 31522 raft_consensus.cc:686] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [term 1 LEADER]: Becoming Leader. State: Replica: 78a6af5f03894a549aca615ced7b723f, State: Running, Role: LEADER
I20260618 22:31:54.504110 31522 consensus_queue.cc:237] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e3b9097b640ddb53c5dffcd3a1ff9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42701 } } peers { permanent_uuid: "178a828519e5482bb9094a1373f967ae" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34597 } } peers { permanent_uuid: "78a6af5f03894a549aca615ced7b723f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 44227 } }
I20260618 22:31:54.506425 31321 raft_consensus.cc:2454] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78a6af5f03894a549aca615ced7b723f in term 1.
I20260618 22:31:54.508993 31050 catalog_manager.cc:5495] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f reported cstate change: term changed from 0 to 1, leader changed from <none> to 78a6af5f03894a549aca615ced7b723f (127.10.242.195). New cstate: current_term: 1 leader_uuid: "78a6af5f03894a549aca615ced7b723f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e3b9097b640ddb53c5dffcd3a1ff9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "178a828519e5482bb9094a1373f967ae" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78a6af5f03894a549aca615ced7b723f" member_type: VOTER last_known_addr { host: "127.10.242.195" port: 44227 } health_report { overall_health: HEALTHY } } }
I20260618 22:31:54.535466 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:54.530691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52564 (local address 127.10.242.195:44227)
0618 22:31:54.530915 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:54.530919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:54.530942 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:31:54.531123 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:54.531127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:54.531203 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:54.531325 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:31:54.531603 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.532313 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.533646 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.534044 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.534137 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:54.534511 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:31:54.534987 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:31:54.535099 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:54.535135 (+ 36us) server_negotiation.cc:299] Negotiation successful
0618 22:31:54.535195 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260618 22:31:54.543727 31321 raft_consensus.cc:1261] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae [term 1 FOLLOWER]: Refusing update from remote peer 78a6af5f03894a549aca615ced7b723f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:31:54.543865 31191 raft_consensus.cc:1261] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9 [term 1 FOLLOWER]: Refusing update from remote peer 78a6af5f03894a549aca615ced7b723f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:31:54.544875 31522 consensus_queue.cc:1035] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [LEADER]: Connected to new peer: Peer: permanent_uuid: "661e3b9097b640ddb53c5dffcd3a1ff9" member_type: VOTER last_known_addr { host: "127.10.242.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:31:54.545209 31521 consensus_queue.cc:1035] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [LEADER]: Connected to new peer: Peer: permanent_uuid: "178a828519e5482bb9094a1373f967ae" member_type: VOTER last_known_addr { host: "127.10.242.194" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:31:54.568939 31050 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:31:54.569267 31050 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:31:54.573108 31050 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 85e71e4eaf29405697abf29de4ee3676: Sending DeleteTablet for 3 replicas of tablet e632d857f1d347309f1bcd16217e44cf
I20260618 22:31:54.574245 31171 tablet_service.cc:1514] Processing DeleteTablet for tablet e632d857f1d347309f1bcd16217e44cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:31:54 UTC) from {username='slave'} at 127.0.0.1:50508
I20260618 22:31:54.574697 31302 tablet_service.cc:1514] Processing DeleteTablet for tablet e632d857f1d347309f1bcd16217e44cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:31:54 UTC) from {username='slave'} at 127.0.0.1:56766
I20260618 22:31:54.575368 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 31109
I20260618 22:31:54.575438 31536 tablet_replica.cc:330] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9: stopping tablet replica
I20260618 22:31:54.575387 31432 tablet_service.cc:1514] Processing DeleteTablet for tablet e632d857f1d347309f1bcd16217e44cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:31:54 UTC) from {username='slave'} at 127.0.0.1:52560
I20260618 22:31:54.575850 31536 raft_consensus.cc:2229] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:31:54.576452 31536 raft_consensus.cc:2258] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:31:54.578787 31536 ts_tablet_manager.cc:1905] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:31:54.586423 31536 ts_tablet_manager.cc:1918] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:31:54.586686 31536 log.cc:1196] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-0/wal/wals/e632d857f1d347309f1bcd16217e44cf
I20260618 22:31:54.587304 31536 ts_tablet_manager.cc:1939] T e632d857f1d347309f1bcd16217e44cf P 661e3b9097b640ddb53c5dffcd3a1ff9: Deleting consensus metadata
I20260618 22:31:54.589478 31034 catalog_manager.cc:4841] TS 661e3b9097b640ddb53c5dffcd3a1ff9 (127.10.242.193:42701): tablet e632d857f1d347309f1bcd16217e44cf (table test-table [id=04b38cba17fd4289a73c62826db4d63a]) successfully deleted
I20260618 22:31:54.590399 31538 tablet_replica.cc:330] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f: stopping tablet replica
I20260618 22:31:54.590932 31538 raft_consensus.cc:2229] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:31:54.591552 31537 tablet_replica.cc:330] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae: stopping tablet replica
I20260618 22:31:54.591993 31537 raft_consensus.cc:2229] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:31:54.592306 31538 raft_consensus.cc:2258] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:31:54.592557 31537 raft_consensus.cc:2258] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:31:54.594607 31537 ts_tablet_manager.cc:1905] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:31:54.594756 31511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:54.576664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50514 (local address 127.10.242.193:42701)
0618 22:31:54.576826 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:54.576831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:54.586914 (+ 10083us) server_negotiation.cc:407] Connection header received
0618 22:31:54.587399 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:54.587405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:54.587519 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:54.587599 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:54.588232 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.589305 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.590994 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.591588 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.591683 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:54.593118 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:54.593131 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:54.593148 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:54.593891 (+ 743us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:54.593965 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:54.593991 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:54.594100 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:54.594306 (+ 206us) server_negotiation.cc:299] Negotiation successful
0618 22:31:54.594419 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260618 22:31:54.595028 31538 ts_tablet_manager.cc:1905] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:31:54.595865 31231 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:54.602433 31537 ts_tablet_manager.cc:1918] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:31:54.602743 31537 log.cc:1196] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-1/wal/wals/e632d857f1d347309f1bcd16217e44cf
I20260618 22:31:54.603377 31537 ts_tablet_manager.cc:1939] T e632d857f1d347309f1bcd16217e44cf P 178a828519e5482bb9094a1373f967ae: Deleting consensus metadata
I20260618 22:31:54.605528 31036 catalog_manager.cc:4841] TS 178a828519e5482bb9094a1373f967ae (127.10.242.194:34597): tablet e632d857f1d347309f1bcd16217e44cf (table test-table [id=04b38cba17fd4289a73c62826db4d63a]) successfully deleted
I20260618 22:31:54.606552 31538 ts_tablet_manager.cc:1918] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:31:54.606737 31538 log.cc:1196] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f: Deleting WAL directory at /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821604371818-11211-0/minicluster-data/ts-2/wal/wals/e632d857f1d347309f1bcd16217e44cf
I20260618 22:31:54.607295 31538 ts_tablet_manager.cc:1939] T e632d857f1d347309f1bcd16217e44cf P 78a6af5f03894a549aca615ced7b723f: Deleting consensus metadata
I20260618 22:31:54.609158 31036 catalog_manager.cc:4841] TS 78a6af5f03894a549aca615ced7b723f (127.10.242.195:44227): tablet e632d857f1d347309f1bcd16217e44cf (table test-table [id=04b38cba17fd4289a73c62826db4d63a]) successfully deleted
I20260618 22:31:54.686303 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 31109
I20260618 22:31:54.707721 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 31240
I20260618 22:31:54.715505 31512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:54.708630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56770 (local address 127.10.242.194:34597)
0618 22:31:54.708820 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:54.708826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:54.708849 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:31:54.708954 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:54.708958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:54.709055 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:54.709177 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:54.709707 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.710532 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.712441 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.712900 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.713002 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:54.714155 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:54.714164 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:54.714179 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:54.714778 (+ 599us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:54.714826 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:54.714846 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:54.714934 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:54.715076 (+ 142us) server_negotiation.cc:299] Negotiation successful
0618 22:31:54.715178 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260618 22:31:54.716276 31362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:54.804766 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 31240
I20260618 22:31:54.818482 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 31370
I20260618 22:31:54.826001 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:54.819294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52568 (local address 127.10.242.195:44227)
0618 22:31:54.819452 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:54.819456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:54.819474 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:31:54.819580 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:54.819584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:54.819669 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:54.819750 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:54.820271 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.821067 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.822877 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:54.823333 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:54.823445 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:54.824562 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:54.824572 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:54.824586 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:54.825235 (+ 649us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:54.825287 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:54.825323 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:54.825401 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:54.825553 (+ 152us) server_negotiation.cc:299] Negotiation successful
0618 22:31:54.825637 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:31:54.826750 31492 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:54.955430 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 31370
I20260618 22:31:54.971840 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 31019
I20260618 22:31:54.972916 31080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:55.081210 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 31019
2026-06-18T22:31:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2669 ms)
[----------] 6 tests from EncryptionPolicyTest (20708 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:31:55 dist-test-slave-0956 krb5kdc[31546](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:31:55 dist-test-slave-0956 krb5kdc[31546](info): set up 2 sockets
Jun 18 22:31:55 dist-test-slave-0956 krb5kdc[31546](info): commencing operation
krb5kdc: starting...
W20260618 22:31:56.105491 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:31:56 dist-test-slave-0956 krb5kdc[31546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:31:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:31:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:31:56.343744 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34675
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:31:56.664978 31559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:56.665292 31559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:56.665395 31559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:56.672732 31559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:31:56.672863 31559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:56.672909 31559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:56.672968 31559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:31:56.673012 31559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:31:56.683058 31559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34675
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:56.684298 31559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:56.685648 31559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:56.697043 31564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:56.697103 31565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:56.697222 31567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:56.697948 31559 server_base.cc:1034] running on GCE node
I20260618 22:31:56.698730 31559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:56.700207 31559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:56.701403 31559 hybrid_clock.cc:648] HybridClock initialized: now 1781821916701356 us; error 45 us; skew 500 ppm
I20260618 22:31:56.701761 31559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:31:56 dist-test-slave-0956 krb5kdc[31546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821916, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:31:56.708633 31559 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:31:56.710237 31559 webserver.cc:460] Webserver started at http://127.0.0.1:33147/ using document root <none> and password file <none>
I20260618 22:31:56.710919 31559 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:56.711036 31559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:56.711320 31559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:56.714380 31559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "4aa42f7ebf134685adcef9cfb99bf7d6"
format_stamp: "Formatted at 2026-06-18 22:31:56 on dist-test-slave-0956"
server_key: "99450b47c3751e3f42adce1ed889505f"
server_key_iv: "cca31f78f7cc7367d2dbd9e92290dec4"
server_key_version: "encryptionkey@0"
I20260618 22:31:56.715131 31559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "4aa42f7ebf134685adcef9cfb99bf7d6"
format_stamp: "Formatted at 2026-06-18 22:31:56 on dist-test-slave-0956"
server_key: "99450b47c3751e3f42adce1ed889505f"
server_key_iv: "cca31f78f7cc7367d2dbd9e92290dec4"
server_key_version: "encryptionkey@0"
I20260618 22:31:56.720273 31559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:31:56.723778 31574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:56.725158 31559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260618 22:31:56.725310 31559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "4aa42f7ebf134685adcef9cfb99bf7d6"
format_stamp: "Formatted at 2026-06-18 22:31:56 on dist-test-slave-0956"
server_key: "99450b47c3751e3f42adce1ed889505f"
server_key_iv: "cca31f78f7cc7367d2dbd9e92290dec4"
server_key_version: "encryptionkey@0"
I20260618 22:31:56.725505 31559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:56.744176 31559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:56.748576 31559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:56.748855 31559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:56.766465 31559 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34675
I20260618 22:31:56.766502 31625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34675 every 8 connection(s)
I20260618 22:31:56.767896 31559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:31:56.771715 31626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:56.776412 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 31559
I20260618 22:31:56.776619 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:31:56.777477 11211 external_mini_cluster.cc:1428] Setting key b36f216de95f34156887e434f2a37a75
I20260618 22:31:56.783444 31626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6: Bootstrap starting.
I20260618 22:31:56.787110 31626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:56.788378 31626 log.cc:824] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:56.791246 31626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6: No bootstrap required, opened a new log
Jun 18 22:31:56 dist-test-slave-0956 krb5kdc[31546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260618 22:31:56.801294 31626 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa42f7ebf134685adcef9cfb99bf7d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34675 } }
I20260618 22:31:56.801766 31626 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:56.801882 31626 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa42f7ebf134685adcef9cfb99bf7d6, State: Initialized, Role: FOLLOWER
I20260618 22:31:56.802091 31629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:56.779411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44793 (local address 127.0.0.1:34675)
0618 22:31:56.780200 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:56.780213 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:56.780257 (+ 44us) server_negotiation.cc:407] Connection header received
0618 22:31:56.782075 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:56.782099 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:56.782528 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:56.782872 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:56.784737 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:56.786219 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:56.788029 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:56.788769 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:56.788976 (+ 207us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:56.792814 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:56.792826 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:56.792845 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:56.792891 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:56.797055 (+ 4164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:56.798030 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:56.798039 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:56.798048 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:56.798308 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:56.798814 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:56.798823 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:56.798827 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:56.799596 (+ 769us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:56.799913 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:56.800346 (+ 433us) server_negotiation.cc:299] Negotiation successful
0618 22:31:56.800686 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":336,"threads_started":1}
I20260618 22:31:56.802561 31626 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa42f7ebf134685adcef9cfb99bf7d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34675 } }
I20260618 22:31:56.802865 31626 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:31:56.803004 31626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:31:56.803172 31626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:56.805835 31626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa42f7ebf134685adcef9cfb99bf7d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34675 } }
I20260618 22:31:56.806111 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:31:56.806373 31626 leader_election.cc:302] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4aa42f7ebf134685adcef9cfb99bf7d6; no voters:
I20260618 22:31:56.806979 31626 leader_election.cc:288] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:31:56.807106 31631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:31:56.808226 31631 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6 [term 1 LEADER]: Becoming Leader. State: Replica: 4aa42f7ebf134685adcef9cfb99bf7d6, State: Running, Role: LEADER
I20260618 22:31:56.808750 31631 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa42f7ebf134685adcef9cfb99bf7d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34675 } }
I20260618 22:31:56.809207 31626 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:31:56.811242 31633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4aa42f7ebf134685adcef9cfb99bf7d6. Latest consensus state: current_term: 1 leader_uuid: "4aa42f7ebf134685adcef9cfb99bf7d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa42f7ebf134685adcef9cfb99bf7d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34675 } } }
I20260618 22:31:56.811626 31633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:31:56.812196 31632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4aa42f7ebf134685adcef9cfb99bf7d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa42f7ebf134685adcef9cfb99bf7d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34675 } } }
I20260618 22:31:56.812475 31632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:31:56.814749 31639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:31:56.820813 31639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:31:56.835111 31639 catalog_manager.cc:1347] Generated new cluster ID: c4c6afc33fba459792fe6757eb6375ab
I20260618 22:31:56.835242 31639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:31:56.847724 31654 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260618 22:31:56.851126 31629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:56.817123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42849 (local address 127.0.0.1:34675)
0618 22:31:56.817301 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:56.817307 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:56.829448 (+ 12141us) server_negotiation.cc:407] Connection header received
0618 22:31:56.829801 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:56.829808 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:56.829897 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:56.829974 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:56.835358 (+ 5384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:56.836725 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:56.838844 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:56.839579 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:56.839683 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:56.843368 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:56.843376 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:56.843384 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:31:56.843433 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:56.847330 (+ 3897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:56.848389 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:56.848395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:56.848398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:56.848635 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:31:56.849236 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:31:56.849242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:31:56.849245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:31:56.849657 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:56.849900 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:56.850561 (+ 661us) server_negotiation.cc:299] Negotiation successful
0618 22:31:56.850743 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:31:56.874540 31639 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:31:56.875846 31639 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:31:56.894891 31639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4aa42f7ebf134685adcef9cfb99bf7d6: Generated new TSK 0
I20260618 22:31:56.896018 31639 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:31:56.907632 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 31559
I20260618 22:31:56.908885 31621 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:31:57.005398 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 31559
W20260618 22:31:57.019240 31544 connection.cc:535] client connection to 127.0.0.1:34675 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34675): Connection reset by peer (error 104)
2026-06-18T22:31:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1920 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:31:57 dist-test-slave-0956 krb5kdc[31658](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:31:57 dist-test-slave-0956 krb5kdc[31658](info): set up 2 sockets
Jun 18 22:31:57 dist-test-slave-0956 krb5kdc[31658](info): commencing operation
krb5kdc: starting...
W20260618 22:31:58.021229 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:31:58 dist-test-slave-0956 krb5kdc[31658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:31:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:31:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:31:58.260257 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35583
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:31:58.580332 31671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:31:58.580663 31671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:31:58.580802 31671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:31:58.588469 31671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:31:58.588595 31671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:31:58.588671 31671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:31:58.588752 31671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:31:58.588811 31671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:31:58.598820 31671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35583
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:31:58.600126 31671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:31:58.601450 31671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:31:58.612957 31679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:58.613026 31676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:31:58.612984 31677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:31:58.613992 31671 server_base.cc:1034] running on GCE node
I20260618 22:31:58.614826 31671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:31:58.616297 31671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:31:58.617515 31671 hybrid_clock.cc:648] HybridClock initialized: now 1781821918617460 us; error 51 us; skew 500 ppm
I20260618 22:31:58.617861 31671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:31:58 dist-test-slave-0956 krb5kdc[31658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821918, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:31:58.620909 31671 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:31:58.622471 31671 webserver.cc:460] Webserver started at http://127.0.0.1:38465/ using document root <none> and password file <none>
I20260618 22:31:58.623153 31671 fs_manager.cc:359] Metadata directory not provided
I20260618 22:31:58.623270 31671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:31:58.623559 31671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:31:58.626542 31671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "7eda7d77c5f74be9aef1e50c72951a30"
format_stamp: "Formatted at 2026-06-18 22:31:58 on dist-test-slave-0956"
server_key: "96e2688058cec41c6d9c6441369421b5"
server_key_iv: "6f8afbd656e418c691e51551bc12a85c"
server_key_version: "encryptionkey@0"
I20260618 22:31:58.627277 31671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "7eda7d77c5f74be9aef1e50c72951a30"
format_stamp: "Formatted at 2026-06-18 22:31:58 on dist-test-slave-0956"
server_key: "96e2688058cec41c6d9c6441369421b5"
server_key_iv: "6f8afbd656e418c691e51551bc12a85c"
server_key_version: "encryptionkey@0"
I20260618 22:31:58.632254 31671 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260618 22:31:58.635740 31686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:31:58.637207 31671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:31:58.637364 31671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "7eda7d77c5f74be9aef1e50c72951a30"
format_stamp: "Formatted at 2026-06-18 22:31:58 on dist-test-slave-0956"
server_key: "96e2688058cec41c6d9c6441369421b5"
server_key_iv: "6f8afbd656e418c691e51551bc12a85c"
server_key_version: "encryptionkey@0"
I20260618 22:31:58.637566 31671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:31:58.654031 31671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:31:58.658013 31671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:31:58.658245 31671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:31:58.675616 31671 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35583
I20260618 22:31:58.675657 31737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35583 every 8 connection(s)
I20260618 22:31:58.677008 31671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:31:58.680868 31738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:31:58.683823 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 31671
I20260618 22:31:58.684005 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:31:58.684523 11211 external_mini_cluster.cc:1428] Setting key bcc842aa72e4ee3647b64e6b1cbe0b9f
I20260618 22:31:58.693905 31738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30: Bootstrap starting.
I20260618 22:31:58.697574 31738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30: Neither blocks nor log segments found. Creating new log.
I20260618 22:31:58.698663 31738 log.cc:824] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30: Log is configured to *not* fsync() on all Append() calls
I20260618 22:31:58.698812 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:58.686516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35051 (local address 127.0.0.1:35583)
0618 22:31:58.687387 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:58.687401 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:58.687447 (+ 46us) server_negotiation.cc:407] Connection header received
0618 22:31:58.689220 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:58.689247 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:58.689708 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:58.690086 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:58.690734 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:58.692150 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:58.694010 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:58.694779 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:58.694918 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:58.696271 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:58.696280 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:58.696295 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:58.696947 (+ 652us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:58.697041 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:58.697112 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:58.697268 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:58.697541 (+ 273us) server_negotiation.cc:299] Negotiation successful
0618 22:31:58.697737 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":394,"threads_started":1}
I20260618 22:31:58.701736 31738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30: No bootstrap required, opened a new log
I20260618 22:31:58.702698 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:31:58.709954 31738 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eda7d77c5f74be9aef1e50c72951a30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35583 } }
I20260618 22:31:58.710465 31738 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:31:58.710599 31738 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eda7d77c5f74be9aef1e50c72951a30, State: Initialized, Role: FOLLOWER
I20260618 22:31:58.711300 31738 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eda7d77c5f74be9aef1e50c72951a30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35583 } }
I20260618 22:31:58.711570 31738 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:31:58.711717 31738 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:31:58.711887 31738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:31:58.714046 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:31:58.707055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47857 (local address 127.0.0.1:35583)
0618 22:31:58.707256 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0618 22:31:58.707262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:31:58.707575 (+ 313us) server_negotiation.cc:407] Connection header received
0618 22:31:58.707734 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:31:58.707738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:31:58.707808 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:31:58.707897 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:31:58.708589 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:58.709592 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:58.711379 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:31:58.711962 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:31:58.712080 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:31:58.713283 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:31:58.713290 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:31:58.713294 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:31:58.713317 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:31:58.713365 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:31:58.713385 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:31:58.713480 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0618 22:31:58.713667 (+ 187us) server_negotiation.cc:299] Negotiation successful
0618 22:31:58.713718 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260618 22:31:58.714942 31738 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eda7d77c5f74be9aef1e50c72951a30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35583 } }
I20260618 22:31:58.715458 31738 leader_election.cc:302] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7eda7d77c5f74be9aef1e50c72951a30; no voters:
I20260618 22:31:58.716039 31738 leader_election.cc:288] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:31:58.716143 31748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:31:58.716782 31748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30 [term 1 LEADER]: Becoming Leader. State: Replica: 7eda7d77c5f74be9aef1e50c72951a30, State: Running, Role: LEADER
I20260618 22:31:58.717319 31748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eda7d77c5f74be9aef1e50c72951a30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35583 } }
I20260618 22:31:58.718083 31738 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:31:58.719996 31749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7eda7d77c5f74be9aef1e50c72951a30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eda7d77c5f74be9aef1e50c72951a30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35583 } } }
I20260618 22:31:58.720439 31749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30 [sys.catalog]: This master's current role is: LEADER
I20260618 22:31:58.720032 31750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7eda7d77c5f74be9aef1e50c72951a30. Latest consensus state: current_term: 1 leader_uuid: "7eda7d77c5f74be9aef1e50c72951a30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eda7d77c5f74be9aef1e50c72951a30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35583 } } }
I20260618 22:31:58.722074 31750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30 [sys.catalog]: This master's current role is: LEADER
W20260618 22:31:58.728626 31764 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:31:58.728742 31764 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:31:58.728883 31754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:31:58.730443 31754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:31:58.738528 31754 catalog_manager.cc:1347] Generated new cluster ID: da50c75298da435a9d0338ce34979edf
I20260618 22:31:58.738704 31754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:31:58.770047 31754 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:31:58.771317 31754 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:31:58.786268 31754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7eda7d77c5f74be9aef1e50c72951a30: Generated new TSK 0
I20260618 22:31:58.787122 31754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:31:58.799504 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 31671
W20260618 22:31:58.800648 31733 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35051
W20260618 22:31:58.801396 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 31671: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:31:58.801596 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 31671
W20260618 22:31:58.816931 31656 connection.cc:535] client connection to 127.0.0.1:35583 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35583): Connection reset by peer (error 104)
2026-06-18T22:31:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1797 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:31:58 dist-test-slave-0956 krb5kdc[31769](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:31:58 dist-test-slave-0956 krb5kdc[31769](info): set up 2 sockets
Jun 18 22:31:58 dist-test-slave-0956 krb5kdc[31769](info): commencing operation
krb5kdc: starting...
W20260618 22:31:59.817225 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:31:59 dist-test-slave-0956 krb5kdc[31769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:31:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:31:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:32:00.062107 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:00.390648 31782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:00.390988 31782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:00.391129 31782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:00.398579 31782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:00.398722 31782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:00.398804 31782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:00.398883 31782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:00.398945 31782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:00.408911 31782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39245
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:00.410270 31782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:00.411602 31782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:00.423554 31788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:00.423586 31787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:00.423519 31790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:00.424307 31782 server_base.cc:1034] running on GCE node
I20260618 22:32:00.425218 31782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:00.426687 31782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:00.427934 31782 hybrid_clock.cc:648] HybridClock initialized: now 1781821920427862 us; error 41 us; skew 500 ppm
I20260618 22:32:00.428292 31782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:00 dist-test-slave-0956 krb5kdc[31769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821920, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:00.431381 31782 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:32:00.432885 31782 webserver.cc:460] Webserver started at http://127.0.0.1:42131/ using document root <none> and password file <none>
I20260618 22:32:00.433604 31782 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:00.433705 31782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:00.434003 31782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:00.436980 31782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "3927af80f5884636a05fd9446f291f24"
format_stamp: "Formatted at 2026-06-18 22:32:00 on dist-test-slave-0956"
server_key: "5c64dd4fe02d516c1e1721c5802d7f1f"
server_key_iv: "e8c8a456159c3a870cd7b6466db45f48"
server_key_version: "encryptionkey@0"
I20260618 22:32:00.437860 31782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "3927af80f5884636a05fd9446f291f24"
format_stamp: "Formatted at 2026-06-18 22:32:00 on dist-test-slave-0956"
server_key: "5c64dd4fe02d516c1e1721c5802d7f1f"
server_key_iv: "e8c8a456159c3a870cd7b6466db45f48"
server_key_version: "encryptionkey@0"
I20260618 22:32:00.443269 31782 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:32:00.446846 31797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:00.448189 31782 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:32:00.448335 31782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "3927af80f5884636a05fd9446f291f24"
format_stamp: "Formatted at 2026-06-18 22:32:00 on dist-test-slave-0956"
server_key: "5c64dd4fe02d516c1e1721c5802d7f1f"
server_key_iv: "e8c8a456159c3a870cd7b6466db45f48"
server_key_version: "encryptionkey@0"
I20260618 22:32:00.448516 31782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:00.454351 31782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:00.458871 31782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:00.459161 31782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:00.476501 31782 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39245
I20260618 22:32:00.476573 31848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39245 every 8 connection(s)
I20260618 22:32:00.477952 31782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:00.481664 31849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:00.485814 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 31782
I20260618 22:32:00.485955 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:32:00.486370 11211 external_mini_cluster.cc:1428] Setting key 764ef765ca077b46343d0befaa075535
I20260618 22:32:00.493790 31849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24: Bootstrap starting.
I20260618 22:32:00.496167 31852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:00.488131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46439 (local address 127.0.0.1:39245)
0618 22:32:00.488903 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:00.488919 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:00.488967 (+ 48us) server_negotiation.cc:407] Connection header received
0618 22:32:00.491874 (+ 2907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:00.491906 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:00.492387 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:00.492850 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:00.494190 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:00.494199 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:00.494216 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:00.494233 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:00.494309 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:00.494342 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:00.494565 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:00.494691 (+ 126us) server_negotiation.cc:299] Negotiation successful
0618 22:32:00.494876 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":306,"threads_started":1}
I20260618 22:32:00.497601 31849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:00.498883 31849 log.cc:824] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:00.500157 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:32:00.502249 31849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24: No bootstrap required, opened a new log
I20260618 22:32:00.507879 31852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:00.505412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50999 (local address 127.0.0.1:39245)
0618 22:32:00.505597 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:00.505603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:00.505838 (+ 235us) server_negotiation.cc:407] Connection header received
0618 22:32:00.505980 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:00.505985 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:00.506085 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:00.506197 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:00.507149 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:00.507155 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:00.507160 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:00.507179 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:00.507231 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:00.507252 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:00.507380 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:00.507520 (+ 140us) server_negotiation.cc:299] Negotiation successful
0618 22:32:00.507581 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:32:00.511456 31849 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3927af80f5884636a05fd9446f291f24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } }
I20260618 22:32:00.511852 31849 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:00.511929 31849 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3927af80f5884636a05fd9446f291f24, State: Initialized, Role: FOLLOWER
I20260618 22:32:00.512434 31849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3927af80f5884636a05fd9446f291f24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } }
I20260618 22:32:00.512650 31849 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:00.512744 31849 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:00.512866 31849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:00.515056 31849 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3927af80f5884636a05fd9446f291f24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } }
I20260618 22:32:00.515596 31849 leader_election.cc:302] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3927af80f5884636a05fd9446f291f24; no voters:
I20260618 22:32:00.516247 31849 leader_election.cc:288] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:00.516371 31859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:00.516963 31859 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24 [term 1 LEADER]: Becoming Leader. State: Replica: 3927af80f5884636a05fd9446f291f24, State: Running, Role: LEADER
I20260618 22:32:00.517572 31859 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3927af80f5884636a05fd9446f291f24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } }
I20260618 22:32:00.518256 31849 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:00.520514 31861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3927af80f5884636a05fd9446f291f24. Latest consensus state: current_term: 1 leader_uuid: "3927af80f5884636a05fd9446f291f24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3927af80f5884636a05fd9446f291f24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } } }
I20260618 22:32:00.520943 31861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:00.520628 31860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3927af80f5884636a05fd9446f291f24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3927af80f5884636a05fd9446f291f24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } } }
I20260618 22:32:00.521193 31860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:00.523219 31868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:00.527997 31868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:00.538079 31868 catalog_manager.cc:1347] Generated new cluster ID: 6a53439af54f4f10aa746d6e611539ba
I20260618 22:32:00.538210 31868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:00.550300 31868 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:00.550868 31868 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:00.569707 31868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3927af80f5884636a05fd9446f291f24: Generated new TSK 0
I20260618 22:32:00.570928 31868 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:32:00.580315 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 31782
W20260618 22:32:00.581583 31844 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46439
W20260618 22:32:00.582476 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 31782: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:32:00.582633 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 31782
2026-06-18T22:32:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1779 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:00 dist-test-slave-0956 krb5kdc[31880](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:00 dist-test-slave-0956 krb5kdc[31880](info): set up 2 sockets
Jun 18 22:32:00 dist-test-slave-0956 krb5kdc[31880](info): commencing operation
krb5kdc: starting...
W20260618 22:32:01.621488 11211 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:32:01 dist-test-slave-0956 krb5kdc[31880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:32:01.870848 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43367
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:02.195386 31893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:02.195689 31893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:02.195824 31893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:02.203007 31893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:02.203119 31893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:02.203193 31893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:02.203284 31893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:02.203336 31893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:02.213145 31893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:02.214418 31893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:02.215716 31893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:02.227056 31898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:02.227209 31899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:02.227346 31901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:02.227730 31893 server_base.cc:1034] running on GCE node
I20260618 22:32:02.228609 31893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:02.230007 31893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:02.231199 31893 hybrid_clock.cc:648] HybridClock initialized: now 1781821922231146 us; error 43 us; skew 500 ppm
I20260618 22:32:02.231523 31893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:02 dist-test-slave-0956 krb5kdc[31880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821922, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:02.239436 31893 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:32:02.240937 31893 webserver.cc:460] Webserver started at http://127.0.0.1:35345/ using document root <none> and password file <none>
I20260618 22:32:02.241679 31893 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:02.241804 31893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:02.242091 31893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:02.245224 31893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "d36289664fed4a6b9abfcb333c752240"
format_stamp: "Formatted at 2026-06-18 22:32:02 on dist-test-slave-0956"
server_key: "52218584eaf4a62a45ad08a49002f6ed"
server_key_iv: "24e318669f664ad2af945f73436aa121"
server_key_version: "encryptionkey@0"
I20260618 22:32:02.245987 31893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "d36289664fed4a6b9abfcb333c752240"
format_stamp: "Formatted at 2026-06-18 22:32:02 on dist-test-slave-0956"
server_key: "52218584eaf4a62a45ad08a49002f6ed"
server_key_iv: "24e318669f664ad2af945f73436aa121"
server_key_version: "encryptionkey@0"
I20260618 22:32:02.251154 31893 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:32:02.254670 31908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:02.255972 31893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:32:02.256147 31893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "d36289664fed4a6b9abfcb333c752240"
format_stamp: "Formatted at 2026-06-18 22:32:02 on dist-test-slave-0956"
server_key: "52218584eaf4a62a45ad08a49002f6ed"
server_key_iv: "24e318669f664ad2af945f73436aa121"
server_key_version: "encryptionkey@0"
I20260618 22:32:02.256327 31893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:02.278712 31893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:02.282793 31893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:02.283041 31893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:02.299901 31893 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43367
I20260618 22:32:02.299927 31959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43367 every 8 connection(s)
I20260618 22:32:02.301263 31893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:02.304283 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 31893
I20260618 22:32:02.304446 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:32:02.304888 11211 external_mini_cluster.cc:1428] Setting key 780bafaec0de8c006f87228eba28dcc7
I20260618 22:32:02.304905 31960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:02.322889 31960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240: Bootstrap starting.
Jun 18 22:32:02 dist-test-slave-0956 krb5kdc[31880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260618 22:32:02.326848 31960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:02.328197 31960 log.cc:824] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:02.331029 31960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240: No bootstrap required, opened a new log
I20260618 22:32:02.332023 31963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:02.306614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34097 (local address 127.0.0.1:43367)
0618 22:32:02.307345 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:02.307360 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:02.307415 (+ 55us) server_negotiation.cc:407] Connection header received
0618 22:32:02.310593 (+ 3178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:02.310623 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:02.311134 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:02.311546 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:02.313949 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:02.316032 (+ 2083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:02.317966 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:02.319121 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:02.319318 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:02.324241 (+ 4923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:02.324252 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:02.324273 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:02.324327 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:02.328057 (+ 3730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:02.328767 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:02.328774 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:02.328780 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:02.328977 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:02.329383 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:02.329387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:02.329389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:02.329901 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:02.330196 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:02.330610 (+ 414us) server_negotiation.cc:299] Negotiation successful
0618 22:32:02.331025 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":291,"threads_started":1}
I20260618 22:32:02.335614 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:32:02.338953 31960 raft_consensus.cc:348] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36289664fed4a6b9abfcb333c752240" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43367 } }
I20260618 22:32:02.339315 31960 raft_consensus.cc:374] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:02.339401 31960 raft_consensus.cc:729] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36289664fed4a6b9abfcb333c752240, State: Initialized, Role: FOLLOWER
I20260618 22:32:02.339995 31960 consensus_queue.cc:260] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36289664fed4a6b9abfcb333c752240" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43367 } }
I20260618 22:32:02.340270 31960 raft_consensus.cc:388] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:02.340423 31960 raft_consensus.cc:482] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:02.340607 31960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:02.343330 31960 raft_consensus.cc:504] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36289664fed4a6b9abfcb333c752240" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43367 } }
I20260618 22:32:02.343880 31960 leader_election.cc:302] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d36289664fed4a6b9abfcb333c752240; no voters:
I20260618 22:32:02.344528 31960 leader_election.cc:288] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:02.344663 31970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:02.346853 31960 sys_catalog.cc:564] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:02.351465 31970 raft_consensus.cc:686] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240 [term 1 LEADER]: Becoming Leader. State: Replica: d36289664fed4a6b9abfcb333c752240, State: Running, Role: LEADER
I20260618 22:32:02.352006 31970 consensus_queue.cc:237] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36289664fed4a6b9abfcb333c752240" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43367 } }
I20260618 22:32:02.354631 31971 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d36289664fed4a6b9abfcb333c752240" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36289664fed4a6b9abfcb333c752240" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43367 } } }
I20260618 22:32:02.355001 31971 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:02.355504 31972 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d36289664fed4a6b9abfcb333c752240. Latest consensus state: current_term: 1 leader_uuid: "d36289664fed4a6b9abfcb333c752240" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36289664fed4a6b9abfcb333c752240" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43367 } } }
I20260618 22:32:02.355794 31972 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:02.356398 31963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:02.339881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38981 (local address 127.0.0.1:43367)
0618 22:32:02.340061 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:02.340067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:02.340086 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:32:02.340227 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:02.340231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:02.340312 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:02.340424 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:02.341804 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:02.342860 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:02.344339 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:02.345147 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:02.345864 (+ 717us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:02.348336 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:02.348342 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:02.348346 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:02.348389 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:02.352107 (+ 3718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:02.353075 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:02.353106 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:02.353110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:02.353266 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:02.353755 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:02.353764 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:02.353768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:02.354233 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:02.355542 (+ 1309us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:02.355736 (+ 194us) server_negotiation.cc:299] Negotiation successful
0618 22:32:02.356004 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:32:02.360695 31977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:02.366415 31977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:02.375710 31977 catalog_manager.cc:1347] Generated new cluster ID: c71b076eae034cd69f0d9a3edad3abd9
I20260618 22:32:02.375854 31977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:02.402783 31977 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:02.404260 31977 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:02.413420 31977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d36289664fed4a6b9abfcb333c752240: Generated new TSK 0
I20260618 22:32:02.414324 31977 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:32:02.421280 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 31893
I20260618 22:32:02.423085 31955 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:02.515164 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 31893
2026-06-18T22:32:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1932 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:02 dist-test-slave-0956 krb5kdc[31992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:02 dist-test-slave-0956 krb5kdc[31992](info): set up 2 sockets
Jun 18 22:32:02 dist-test-slave-0956 krb5kdc[31992](info): commencing operation
krb5kdc: starting...
W20260618 22:32:03.537364 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:32:03 dist-test-slave-0956 krb5kdc[31992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:32:03.787818 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43961
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:04.114197 32005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:04.114503 32005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:04.114629 32005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:04.122190 32005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:04.122323 32005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:04.122385 32005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:04.122466 32005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:04.122521 32005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:04.132723 32005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:04.134071 32005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:04.135380 32005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:04.146776 32010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:04.146755 32013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:04.146754 32011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:04.147457 32005 server_base.cc:1034] running on GCE node
I20260618 22:32:04.148334 32005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:04.149840 32005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:04.151044 32005 hybrid_clock.cc:648] HybridClock initialized: now 1781821924150981 us; error 51 us; skew 500 ppm
I20260618 22:32:04.151387 32005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:04 dist-test-slave-0956 krb5kdc[31992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821924, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:04.154513 32005 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:32:04.155992 32005 webserver.cc:460] Webserver started at http://127.0.0.1:37511/ using document root <none> and password file <none>
I20260618 22:32:04.156666 32005 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:04.156770 32005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:04.157059 32005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:04.160099 32005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "8e31046f286044038698f4a1d7674c13"
format_stamp: "Formatted at 2026-06-18 22:32:04 on dist-test-slave-0956"
server_key: "2b26192d0df61995f1d1609de8abdcd7"
server_key_iv: "a5dcdb5f4d75b534d82a3ce40bb53330"
server_key_version: "encryptionkey@0"
I20260618 22:32:04.160861 32005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "8e31046f286044038698f4a1d7674c13"
format_stamp: "Formatted at 2026-06-18 22:32:04 on dist-test-slave-0956"
server_key: "2b26192d0df61995f1d1609de8abdcd7"
server_key_iv: "a5dcdb5f4d75b534d82a3ce40bb53330"
server_key_version: "encryptionkey@0"
I20260618 22:32:04.166091 32005 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260618 22:32:04.169548 32020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:04.171053 32005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260618 22:32:04.171249 32005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "8e31046f286044038698f4a1d7674c13"
format_stamp: "Formatted at 2026-06-18 22:32:04 on dist-test-slave-0956"
server_key: "2b26192d0df61995f1d1609de8abdcd7"
server_key_iv: "a5dcdb5f4d75b534d82a3ce40bb53330"
server_key_version: "encryptionkey@0"
I20260618 22:32:04.171456 32005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:04.183786 32005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:04.188341 32005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:04.188630 32005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:04.205964 32005 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43961
I20260618 22:32:04.205978 32071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43961 every 8 connection(s)
I20260618 22:32:04.207389 32005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:04.211118 32072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:04.212023 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 32005
I20260618 22:32:04.212191 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:32:04.212577 11211 external_mini_cluster.cc:1428] Setting key 010c330727dc33bfdbfb4ab7c281f6fd
I20260618 22:32:04.225641 32072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13: Bootstrap starting.
I20260618 22:32:04.227877 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:04.214495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57049 (local address 127.0.0.1:43961)
0618 22:32:04.215356 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:04.215371 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:04.215426 (+ 55us) server_negotiation.cc:407] Connection header received
0618 22:32:04.217518 (+ 2092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:04.217551 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:04.218114 (+ 563us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:04.218610 (+ 496us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:04.219164 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:04.220971 (+ 1807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:04.222436 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:04.223561 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:04.223738 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:04.224852 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:04.224864 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:04.224885 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:04.225663 (+ 778us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:04.225776 (+ 113us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:04.225809 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:04.225918 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:04.226195 (+ 277us) server_negotiation.cc:299] Negotiation successful
0618 22:32:04.226481 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":373,"threads_started":1}
I20260618 22:32:04.229633 32072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:04.230835 32072 log.cc:824] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:04.231910 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:32:04.234057 32072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13: No bootstrap required, opened a new log
I20260618 22:32:04.243588 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:04.237014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45393 (local address 127.0.0.1:43961)
0618 22:32:04.237254 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:04.237266 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:04.237726 (+ 460us) server_negotiation.cc:407] Connection header received
0618 22:32:04.237885 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:04.237890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:04.237969 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:04.238087 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:04.238548 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:04.239628 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:04.241053 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:04.241789 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:04.241919 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:04.242905 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:04.242911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:04.242915 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:04.242937 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:04.242984 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:04.243003 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:04.243092 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:04.243188 (+ 96us) server_negotiation.cc:299] Negotiation successful
0618 22:32:04.243283 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:32:04.243571 32072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e31046f286044038698f4a1d7674c13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43961 } }
I20260618 22:32:04.244062 32072 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:04.244163 32072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e31046f286044038698f4a1d7674c13, State: Initialized, Role: FOLLOWER
I20260618 22:32:04.244814 32072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e31046f286044038698f4a1d7674c13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43961 } }
I20260618 22:32:04.245141 32072 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:04.245283 32072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:04.245438 32072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:04.247742 32072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e31046f286044038698f4a1d7674c13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43961 } }
I20260618 22:32:04.248332 32072 leader_election.cc:302] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e31046f286044038698f4a1d7674c13; no voters:
I20260618 22:32:04.248958 32072 leader_election.cc:288] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:04.249158 32082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:04.249889 32082 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13 [term 1 LEADER]: Becoming Leader. State: Replica: 8e31046f286044038698f4a1d7674c13, State: Running, Role: LEADER
I20260618 22:32:04.250494 32082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e31046f286044038698f4a1d7674c13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43961 } }
I20260618 22:32:04.251281 32072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:04.253152 32083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e31046f286044038698f4a1d7674c13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e31046f286044038698f4a1d7674c13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43961 } } }
I20260618 22:32:04.253665 32083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:04.253247 32084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e31046f286044038698f4a1d7674c13. Latest consensus state: current_term: 1 leader_uuid: "8e31046f286044038698f4a1d7674c13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e31046f286044038698f4a1d7674c13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43961 } } }
I20260618 22:32:04.254117 32084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:04.254490 32089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:04.264741 32089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:04.275002 32089 catalog_manager.cc:1347] Generated new cluster ID: d1de34373f6b4b8a91f832173722fc42
I20260618 22:32:04.275118 32089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:04.293159 32089 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:04.295151 32089 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:04.309731 32089 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e31046f286044038698f4a1d7674c13: Generated new TSK 0
I20260618 22:32:04.310868 32089 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:32:04.320047 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 32005
W20260618 22:32:04.321313 32067 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57049
W20260618 22:32:04.322132 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 32005: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:32:04.322275 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 32005
2026-06-18T22:32:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1806 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:04 dist-test-slave-0956 krb5kdc[32103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:04 dist-test-slave-0956 krb5kdc[32103](info): set up 2 sockets
Jun 18 22:32:04 dist-test-slave-0956 krb5kdc[32103](info): commencing operation
krb5kdc: starting...
W20260618 22:32:05.337191 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:32:05 dist-test-slave-0956 krb5kdc[32103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:32:05.580847 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39815
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:05.903262 32116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:05.903575 32116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:05.903729 32116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:05.911139 32116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:05.911283 32116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:05.911366 32116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:05.911456 32116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:05.911514 32116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:05.921763 32116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:05.923130 32116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:05.924467 32116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:05.936409 32122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:05.936430 32124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:05.936482 32121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:05.937461 32116 server_base.cc:1034] running on GCE node
I20260618 22:32:05.938246 32116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:05.939683 32116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:05.940902 32116 hybrid_clock.cc:648] HybridClock initialized: now 1781821925940854 us; error 32 us; skew 500 ppm
I20260618 22:32:05.941293 32116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:05 dist-test-slave-0956 krb5kdc[32103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821925, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:05.944343 32116 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:32:05.945938 32116 webserver.cc:460] Webserver started at http://127.0.0.1:43687/ using document root <none> and password file <none>
I20260618 22:32:05.946609 32116 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:05.946714 32116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:05.947006 32116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:05.950165 32116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "9ba624b27f634f60896e2b864fe0ac6b"
format_stamp: "Formatted at 2026-06-18 22:32:05 on dist-test-slave-0956"
server_key: "c07ea90eb31d0f4279cacf787c6c6cd9"
server_key_iv: "312fa0cd0510bac5152d15fe8ee2a850"
server_key_version: "encryptionkey@0"
I20260618 22:32:05.950999 32116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "9ba624b27f634f60896e2b864fe0ac6b"
format_stamp: "Formatted at 2026-06-18 22:32:05 on dist-test-slave-0956"
server_key: "c07ea90eb31d0f4279cacf787c6c6cd9"
server_key_iv: "312fa0cd0510bac5152d15fe8ee2a850"
server_key_version: "encryptionkey@0"
I20260618 22:32:05.956360 32116 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:32:05.959988 32131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:05.961548 32116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:32:05.961736 32116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "9ba624b27f634f60896e2b864fe0ac6b"
format_stamp: "Formatted at 2026-06-18 22:32:05 on dist-test-slave-0956"
server_key: "c07ea90eb31d0f4279cacf787c6c6cd9"
server_key_iv: "312fa0cd0510bac5152d15fe8ee2a850"
server_key_version: "encryptionkey@0"
I20260618 22:32:05.961941 32116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:05.967746 32116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:05.972147 32116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:05.972443 32116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:05.989097 32116 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39815
I20260618 22:32:05.989144 32182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39815 every 8 connection(s)
I20260618 22:32:05.990484 32116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:05.993543 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 32116
I20260618 22:32:05.993711 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:32:05.994131 11211 external_mini_cluster.cc:1428] Setting key ea5483249937256853e0e552564646f3
I20260618 22:32:05.994354 32183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:06.003360 32186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:05.995905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50849 (local address 127.0.0.1:39815)
0618 22:32:05.996707 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:05.996733 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:05.996780 (+ 47us) server_negotiation.cc:407] Connection header received
0618 22:32:05.999229 (+ 2449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:05.999259 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:05.999767 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:06.000264 (+ 497us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:06.001286 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:06.001294 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:06.001312 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:06.001329 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:06.001406 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:06.001434 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:06.001653 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:06.002227 (+ 574us) server_negotiation.cc:299] Negotiation successful
0618 22:32:06.002398 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":325,"threads_started":1}
I20260618 22:32:06.007405 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:32:06.008705 32183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b: Bootstrap starting.
I20260618 22:32:06.012178 32183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:06.013481 32183 log.cc:824] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:06.013963 32186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:06.011558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36907 (local address 127.0.0.1:39815)
0618 22:32:06.011735 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:06.011741 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:06.012024 (+ 283us) server_negotiation.cc:407] Connection header received
0618 22:32:06.012193 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:06.012196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:06.012250 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:06.012340 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:06.013327 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:06.013331 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:06.013334 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:06.013348 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:06.013385 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:06.013400 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:06.013488 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:06.013606 (+ 118us) server_negotiation.cc:299] Negotiation successful
0618 22:32:06.013670 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:32:06.016310 32183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b: No bootstrap required, opened a new log
I20260618 22:32:06.023643 32183 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba624b27f634f60896e2b864fe0ac6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39815 } }
I20260618 22:32:06.024154 32183 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:06.024276 32183 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ba624b27f634f60896e2b864fe0ac6b, State: Initialized, Role: FOLLOWER
I20260618 22:32:06.024917 32183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba624b27f634f60896e2b864fe0ac6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39815 } }
I20260618 22:32:06.025200 32183 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:06.025345 32183 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:06.025497 32183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:06.027698 32183 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba624b27f634f60896e2b864fe0ac6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39815 } }
I20260618 22:32:06.028187 32183 leader_election.cc:302] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ba624b27f634f60896e2b864fe0ac6b; no voters:
I20260618 22:32:06.028733 32183 leader_election.cc:288] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:06.028864 32193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:06.029510 32193 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b [term 1 LEADER]: Becoming Leader. State: Replica: 9ba624b27f634f60896e2b864fe0ac6b, State: Running, Role: LEADER
I20260618 22:32:06.030084 32193 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba624b27f634f60896e2b864fe0ac6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39815 } }
I20260618 22:32:06.030836 32183 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:06.032949 32194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ba624b27f634f60896e2b864fe0ac6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba624b27f634f60896e2b864fe0ac6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39815 } } }
I20260618 22:32:06.033398 32194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:06.033398 32195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ba624b27f634f60896e2b864fe0ac6b. Latest consensus state: current_term: 1 leader_uuid: "9ba624b27f634f60896e2b864fe0ac6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba624b27f634f60896e2b864fe0ac6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39815 } } }
I20260618 22:32:06.033682 32195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:06.035156 32202 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:06.039438 32202 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:06.048502 32202 catalog_manager.cc:1347] Generated new cluster ID: 40230d8f0d7e41878615650729bd5bd6
I20260618 22:32:06.048635 32202 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:06.067929 32202 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:06.068378 32202 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:06.077132 32202 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ba624b27f634f60896e2b864fe0ac6b: Generated new TSK 0
I20260618 22:32:06.078280 32202 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:32:06.094951 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 32116
W20260618 22:32:06.096169 32178 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50849
W20260618 22:32:06.096958 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 32116: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:32:06.097138 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 32116
2026-06-18T22:32:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1774 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:06 dist-test-slave-0956 krb5kdc[32214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:06 dist-test-slave-0956 krb5kdc[32214](info): set up 2 sockets
Jun 18 22:32:06 dist-test-slave-0956 krb5kdc[32214](info): commencing operation
krb5kdc: starting...
W20260618 22:32:07.121213 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:32:07 dist-test-slave-0956 krb5kdc[32214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:32:07.364329 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33867
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:07.687424 32227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:07.687758 32227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:07.687887 32227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:07.695271 32227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:07.695393 32227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:07.695467 32227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:07.695547 32227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:07.695604 32227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:07.705719 32227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:07.707120 32227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:07.708434 32227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:07.720300 32235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:07.720321 32233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:07.720985 32232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:07.721429 32227 server_base.cc:1034] running on GCE node
I20260618 22:32:07.722324 32227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:07.724064 32227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:07.725274 32227 hybrid_clock.cc:648] HybridClock initialized: now 1781821927725228 us; error 40 us; skew 500 ppm
I20260618 22:32:07.725641 32227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:07 dist-test-slave-0956 krb5kdc[32214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821927, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:07.737617 32227 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:32:07.739355 32227 webserver.cc:460] Webserver started at http://127.0.0.1:34147/ using document root <none> and password file <none>
I20260618 22:32:07.740074 32227 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:07.740207 32227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:07.740509 32227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:07.743705 32227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "58f75e0b5de34e1bba4b5e4ca6747ea0"
format_stamp: "Formatted at 2026-06-18 22:32:07 on dist-test-slave-0956"
server_key: "b2afa2705482f41235a89098c31ddc29"
server_key_iv: "f24c15ddab826e53b8fb9256a587ef12"
server_key_version: "encryptionkey@0"
I20260618 22:32:07.744534 32227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "58f75e0b5de34e1bba4b5e4ca6747ea0"
format_stamp: "Formatted at 2026-06-18 22:32:07 on dist-test-slave-0956"
server_key: "b2afa2705482f41235a89098c31ddc29"
server_key_iv: "f24c15ddab826e53b8fb9256a587ef12"
server_key_version: "encryptionkey@0"
I20260618 22:32:07.749997 32227 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260618 22:32:07.754056 32242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:07.755740 32227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:32:07.755931 32227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "58f75e0b5de34e1bba4b5e4ca6747ea0"
format_stamp: "Formatted at 2026-06-18 22:32:07 on dist-test-slave-0956"
server_key: "b2afa2705482f41235a89098c31ddc29"
server_key_iv: "f24c15ddab826e53b8fb9256a587ef12"
server_key_version: "encryptionkey@0"
I20260618 22:32:07.756141 32227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:07.783772 32227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:07.788239 32227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:07.788520 32227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:07.806303 32227 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33867
I20260618 22:32:07.806335 32293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33867 every 8 connection(s)
I20260618 22:32:07.807806 32227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:07.811635 32294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:07.817793 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 32227
I20260618 22:32:07.817986 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:32:07.818523 11211 external_mini_cluster.cc:1428] Setting key 9885885a7ea8de381f82bab2e937f603
I20260618 22:32:07.823555 32294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0: Bootstrap starting.
I20260618 22:32:07.827396 32294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:07.828815 32294 log.cc:824] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:07.832572 32294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0: No bootstrap required, opened a new log
Jun 18 22:32:07 dist-test-slave-0956 krb5kdc[32214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260618 22:32:07.841646 32294 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f75e0b5de34e1bba4b5e4ca6747ea0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33867 } }
I20260618 22:32:07.842072 32294 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:07.842188 32294 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f75e0b5de34e1bba4b5e4ca6747ea0, State: Initialized, Role: FOLLOWER
I20260618 22:32:07.842947 32294 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f75e0b5de34e1bba4b5e4ca6747ea0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33867 } }
I20260618 22:32:07.843199 32294 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:07.843334 32294 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:07.843494 32294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:07.844339 32297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:07.820615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:46065 (local address 127.0.0.1:33867)
0618 22:32:07.821526 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:07.821540 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:07.821584 (+ 44us) server_negotiation.cc:407] Connection header received
0618 22:32:07.823337 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:07.823364 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:07.823858 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:07.824252 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:07.826116 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:07.827556 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:07.829410 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:07.830573 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:07.830740 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:07.835950 (+ 5210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:07.835960 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:07.835982 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:07.836029 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:07.839349 (+ 3320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:07.840523 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:07.840531 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:07.840539 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:07.840747 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:07.841286 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:07.841290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:07.841292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:07.841884 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:07.842224 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:07.842719 (+ 495us) server_negotiation.cc:299] Negotiation successful
0618 22:32:07.843075 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":439,"threads_started":1}
I20260618 22:32:07.846690 32294 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f75e0b5de34e1bba4b5e4ca6747ea0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33867 } }
I20260618 22:32:07.847388 32294 leader_election.cc:302] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58f75e0b5de34e1bba4b5e4ca6747ea0; no voters:
I20260618 22:32:07.848269 32294 leader_election.cc:288] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:07.848410 32299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:07.849584 32299 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0 [term 1 LEADER]: Becoming Leader. State: Replica: 58f75e0b5de34e1bba4b5e4ca6747ea0, State: Running, Role: LEADER
I20260618 22:32:07.850337 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:32:07.850193 32299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f75e0b5de34e1bba4b5e4ca6747ea0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33867 } }
I20260618 22:32:07.850873 32294 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:07.854382 32300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58f75e0b5de34e1bba4b5e4ca6747ea0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f75e0b5de34e1bba4b5e4ca6747ea0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33867 } } }
I20260618 22:32:07.854887 32300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:07.869323 32304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:07.869611 32301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58f75e0b5de34e1bba4b5e4ca6747ea0. Latest consensus state: current_term: 1 leader_uuid: "58f75e0b5de34e1bba4b5e4ca6747ea0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f75e0b5de34e1bba4b5e4ca6747ea0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33867 } } }
I20260618 22:32:07.869947 32301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:07.875552 32304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:07.884336 32297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:07.867368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:47265 (local address 127.0.0.1:33867)
0618 22:32:07.867581 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:07.867587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:07.867606 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:32:07.867683 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:07.867693 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:07.867775 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:07.867876 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:07.869896 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:07.871035 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:07.872889 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:07.873568 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:07.873693 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:07.877208 (+ 3515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:07.877218 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:07.877223 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:07.877267 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:07.881213 (+ 3946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:07.882157 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:07.882163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:07.882167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:07.882295 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:07.882759 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:07.882764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:07.882768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:07.883151 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:07.883424 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:07.883784 (+ 360us) server_negotiation.cc:299] Negotiation successful
0618 22:32:07.883981 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:32:07.895660 32304 catalog_manager.cc:1347] Generated new cluster ID: cfdf06f9ca284fe29016a869210d1a3e
I20260618 22:32:07.895800 32304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:07.917559 32304 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:07.918958 32304 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:07.928002 32304 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58f75e0b5de34e1bba4b5e4ca6747ea0: Generated new TSK 0
I20260618 22:32:07.928946 32304 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:32:07.936187 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 32227
I20260618 22:32:07.937451 32289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:08.030313 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 32227
W20260618 22:32:08.044620 32212 connection.cc:535] client connection to 127.0.0.1:33867 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33867): Connection reset by peer (error 104)
2026-06-18T22:32:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1934 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:08 dist-test-slave-0956 krb5kdc[32326](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:08 dist-test-slave-0956 krb5kdc[32326](info): set up 2 sockets
Jun 18 22:32:08 dist-test-slave-0956 krb5kdc[32326](info): commencing operation
krb5kdc: starting...
W20260618 22:32:09.049369 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:32:09 dist-test-slave-0956 krb5kdc[32326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:32:09.296897 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37125
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:09.621479 32339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:09.621816 32339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:09.621958 32339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:09.629354 32339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:09.629488 32339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:09.629560 32339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:09.629642 32339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:09.629710 32339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:09.640149 32339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:09.641523 32339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:09.642855 32339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:09.654764 32344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:09.654793 32345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:09.654999 32347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:09.656132 32339 server_base.cc:1034] running on GCE node
I20260618 22:32:09.657002 32339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:09.658561 32339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:09.659792 32339 hybrid_clock.cc:648] HybridClock initialized: now 1781821929659736 us; error 46 us; skew 500 ppm
I20260618 22:32:09.660163 32339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:09 dist-test-slave-0956 krb5kdc[32326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821929, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:09.663470 32339 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:32:09.664996 32339 webserver.cc:460] Webserver started at http://127.0.0.1:41759/ using document root <none> and password file <none>
I20260618 22:32:09.665706 32339 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:09.665830 32339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:09.666133 32339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:09.669225 32339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "c608f03b0379427aaf6f7ceff4840a31"
format_stamp: "Formatted at 2026-06-18 22:32:09 on dist-test-slave-0956"
server_key: "a4778b0b48781bcd6bb07f4738b3c627"
server_key_iv: "159ef42c4de3cb35c8789574f530fad2"
server_key_version: "encryptionkey@0"
I20260618 22:32:09.669991 32339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "c608f03b0379427aaf6f7ceff4840a31"
format_stamp: "Formatted at 2026-06-18 22:32:09 on dist-test-slave-0956"
server_key: "a4778b0b48781bcd6bb07f4738b3c627"
server_key_iv: "159ef42c4de3cb35c8789574f530fad2"
server_key_version: "encryptionkey@0"
I20260618 22:32:09.675292 32339 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:32:09.679307 32354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:09.681046 32339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:32:09.681285 32339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "c608f03b0379427aaf6f7ceff4840a31"
format_stamp: "Formatted at 2026-06-18 22:32:09 on dist-test-slave-0956"
server_key: "a4778b0b48781bcd6bb07f4738b3c627"
server_key_iv: "159ef42c4de3cb35c8789574f530fad2"
server_key_version: "encryptionkey@0"
I20260618 22:32:09.681496 32339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:09.696084 32339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:09.700340 32339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:09.700627 32339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:09.718241 32339 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37125
I20260618 22:32:09.718289 32405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37125 every 8 connection(s)
I20260618 22:32:09.719640 32339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:09.721153 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 32339
I20260618 22:32:09.721318 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:32:09.721870 11211 external_mini_cluster.cc:1428] Setting key 8e5da121625231e7419a556d1299ec0d
I20260618 22:32:09.724443 32406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:09.737335 32409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:09.723676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:46325 (local address 127.0.0.1:37125)
0618 22:32:09.724465 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:09.724482 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:09.724536 (+ 54us) server_negotiation.cc:407] Connection header received
0618 22:32:09.726779 (+ 2243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:09.726820 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:09.727362 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:09.727841 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:09.728376 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:09.730408 (+ 2032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:09.731814 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:09.732840 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:09.733026 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:09.734339 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:09.734350 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:09.734372 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:09.735133 (+ 761us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:09.735235 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:09.735268 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:09.735458 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:09.735713 (+ 255us) server_negotiation.cc:299] Negotiation successful
0618 22:32:09.735913 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":313,"threads_started":1}
I20260618 22:32:09.738603 32406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31: Bootstrap starting.
I20260618 22:32:09.741287 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:32:09.741945 32406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:09.743199 32406 log.cc:824] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:09.746690 32406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31: No bootstrap required, opened a new log
I20260618 22:32:09.752729 32409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:09.746412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:41877 (local address 127.0.0.1:37125)
0618 22:32:09.746634 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:09.746641 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:09.747133 (+ 492us) server_negotiation.cc:407] Connection header received
0618 22:32:09.747306 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:09.747310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:09.747378 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:09.747460 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:09.747996 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:09.748787 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:09.750217 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:09.750658 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:09.750773 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:09.751976 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:09.751981 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:09.751984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:09.752004 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:09.752063 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:09.752082 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:09.752189 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:09.752381 (+ 192us) server_negotiation.cc:299] Negotiation successful
0618 22:32:09.752436 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:32:09.755734 32406 raft_consensus.cc:348] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c608f03b0379427aaf6f7ceff4840a31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37125 } }
I20260618 22:32:09.756191 32406 raft_consensus.cc:374] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:09.756300 32406 raft_consensus.cc:729] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c608f03b0379427aaf6f7ceff4840a31, State: Initialized, Role: FOLLOWER
I20260618 22:32:09.756943 32406 consensus_queue.cc:260] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c608f03b0379427aaf6f7ceff4840a31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37125 } }
I20260618 22:32:09.757243 32406 raft_consensus.cc:388] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:09.757403 32406 raft_consensus.cc:482] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:09.757584 32406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:09.760519 32406 raft_consensus.cc:504] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c608f03b0379427aaf6f7ceff4840a31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37125 } }
I20260618 22:32:09.761062 32406 leader_election.cc:302] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c608f03b0379427aaf6f7ceff4840a31; no voters:
I20260618 22:32:09.761772 32406 leader_election.cc:288] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:09.761915 32416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:09.762648 32416 raft_consensus.cc:686] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31 [term 1 LEADER]: Becoming Leader. State: Replica: c608f03b0379427aaf6f7ceff4840a31, State: Running, Role: LEADER
I20260618 22:32:09.763193 32416 consensus_queue.cc:237] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c608f03b0379427aaf6f7ceff4840a31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37125 } }
I20260618 22:32:09.764051 32406 sys_catalog.cc:564] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:09.765695 32418 sys_catalog.cc:455] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c608f03b0379427aaf6f7ceff4840a31. Latest consensus state: current_term: 1 leader_uuid: "c608f03b0379427aaf6f7ceff4840a31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c608f03b0379427aaf6f7ceff4840a31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37125 } } }
I20260618 22:32:09.766062 32418 sys_catalog.cc:458] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:09.769318 32417 sys_catalog.cc:455] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c608f03b0379427aaf6f7ceff4840a31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c608f03b0379427aaf6f7ceff4840a31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37125 } } }
I20260618 22:32:09.769678 32417 sys_catalog.cc:458] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:09.770272 32421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:09.774923 32421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:09.783152 32421 catalog_manager.cc:1347] Generated new cluster ID: 9f7338265cf945c798dc63ff782db7ca
I20260618 22:32:09.783268 32421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:09.796990 32421 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:09.798285 32421 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:09.808686 32421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c608f03b0379427aaf6f7ceff4840a31: Generated new TSK 0
I20260618 22:32:09.809592 32421 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:32:09.817924 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 32339
W20260618 22:32:09.819870 32401 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.84:46325
W20260618 22:32:09.820591 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 32339: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:32:09.820745 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 32339
W20260618 22:32:09.834385 32324 connection.cc:535] client connection to 127.0.0.1:37125 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37125): Connection reset by peer (error 104)
2026-06-18T22:32:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1789 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:09 dist-test-slave-0956 krb5kdc[32437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:09 dist-test-slave-0956 krb5kdc[32437](info): set up 2 sockets
Jun 18 22:32:09 dist-test-slave-0956 krb5kdc[32437](info): commencing operation
krb5kdc: starting...
W20260618 22:32:10.841320 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.988s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:32:10 dist-test-slave-0956 krb5kdc[32437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:32:11.092140 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44285
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:11.428694 32450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:11.429003 32450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:11.429147 32450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:11.436488 32450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:11.436617 32450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:11.436672 32450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:11.436756 32450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:11.436812 32450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:11.447410 32450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:11.448768 32450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:11.450111 32450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:11.461575 32455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:11.461591 32458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:11.461637 32456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:11.462340 32450 server_base.cc:1034] running on GCE node
I20260618 22:32:11.463179 32450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:11.464648 32450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:11.465862 32450 hybrid_clock.cc:648] HybridClock initialized: now 1781821931465810 us; error 42 us; skew 500 ppm
I20260618 22:32:11.466205 32450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:11 dist-test-slave-0956 krb5kdc[32437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821931, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:11.469364 32450 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:32:11.470829 32450 webserver.cc:460] Webserver started at http://127.0.0.1:39199/ using document root <none> and password file <none>
I20260618 22:32:11.471509 32450 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:11.471628 32450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:11.471915 32450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:11.475070 32450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "280ef0c29f8d4b3283f7c3c91ea78108"
format_stamp: "Formatted at 2026-06-18 22:32:11 on dist-test-slave-0956"
server_key: "dff7a0a1c6fe944c7165396fd0e0780d"
server_key_iv: "814faf810f07334ea5bc6fef55981a01"
server_key_version: "encryptionkey@0"
I20260618 22:32:11.475845 32450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "280ef0c29f8d4b3283f7c3c91ea78108"
format_stamp: "Formatted at 2026-06-18 22:32:11 on dist-test-slave-0956"
server_key: "dff7a0a1c6fe944c7165396fd0e0780d"
server_key_iv: "814faf810f07334ea5bc6fef55981a01"
server_key_version: "encryptionkey@0"
I20260618 22:32:11.481212 32450 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260618 22:32:11.484745 32465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:11.486155 32450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:32:11.486362 32450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "280ef0c29f8d4b3283f7c3c91ea78108"
format_stamp: "Formatted at 2026-06-18 22:32:11 on dist-test-slave-0956"
server_key: "dff7a0a1c6fe944c7165396fd0e0780d"
server_key_iv: "814faf810f07334ea5bc6fef55981a01"
server_key_version: "encryptionkey@0"
I20260618 22:32:11.486554 32450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:11.492444 32450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:11.496641 32450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:11.496924 32450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:11.514393 32450 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44285
I20260618 22:32:11.514421 32516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44285 every 8 connection(s)
I20260618 22:32:11.515919 32450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:11.519727 32517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:11.525553 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 32450
I20260618 22:32:11.525712 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:32:11.526146 11211 external_mini_cluster.cc:1428] Setting key f5dd8a8becd4be665b4f1345faca5227
I20260618 22:32:11.531000 32517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108: Bootstrap starting.
I20260618 22:32:11.534250 32520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:11.528068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:43465 (local address 127.0.0.1:44285)
0618 22:32:11.528892 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:11.528908 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:11.528953 (+ 45us) server_negotiation.cc:407] Connection header received
0618 22:32:11.530834 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:11.530865 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:11.531285 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:11.531651 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:11.532656 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:11.532663 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:11.532675 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:11.532699 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:11.532763 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:11.532784 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:11.532991 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:11.533158 (+ 167us) server_negotiation.cc:299] Negotiation successful
0618 22:32:11.533340 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":359,"threads_started":1}
I20260618 22:32:11.534286 32517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:11.535630 32517 log.cc:824] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:11.538275 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:32:11.538694 32517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108: No bootstrap required, opened a new log
I20260618 22:32:11.544708 32520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:11.542406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:41363 (local address 127.0.0.1:44285)
0618 22:32:11.542584 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:11.542590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:11.542894 (+ 304us) server_negotiation.cc:407] Connection header received
0618 22:32:11.543027 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:11.543032 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:11.543091 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:11.543203 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:11.544081 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:11.544087 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:11.544092 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:11.544111 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:11.544165 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:11.544189 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:11.544309 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:11.544358 (+ 49us) server_negotiation.cc:299] Negotiation successful
0618 22:32:11.544415 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:32:11.547441 32517 raft_consensus.cc:348] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "280ef0c29f8d4b3283f7c3c91ea78108" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44285 } }
I20260618 22:32:11.547813 32517 raft_consensus.cc:374] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:11.547889 32517 raft_consensus.cc:729] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 280ef0c29f8d4b3283f7c3c91ea78108, State: Initialized, Role: FOLLOWER
I20260618 22:32:11.548448 32517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "280ef0c29f8d4b3283f7c3c91ea78108" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44285 } }
I20260618 22:32:11.548666 32517 raft_consensus.cc:388] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:11.548763 32517 raft_consensus.cc:482] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:11.548903 32517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:11.551388 32517 raft_consensus.cc:504] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "280ef0c29f8d4b3283f7c3c91ea78108" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44285 } }
I20260618 22:32:11.551972 32517 leader_election.cc:302] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 280ef0c29f8d4b3283f7c3c91ea78108; no voters:
I20260618 22:32:11.552490 32517 leader_election.cc:288] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:11.552624 32527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:11.553285 32527 raft_consensus.cc:686] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108 [term 1 LEADER]: Becoming Leader. State: Replica: 280ef0c29f8d4b3283f7c3c91ea78108, State: Running, Role: LEADER
I20260618 22:32:11.553870 32527 consensus_queue.cc:237] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "280ef0c29f8d4b3283f7c3c91ea78108" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44285 } }
I20260618 22:32:11.554647 32517 sys_catalog.cc:564] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:11.557219 32529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 280ef0c29f8d4b3283f7c3c91ea78108. Latest consensus state: current_term: 1 leader_uuid: "280ef0c29f8d4b3283f7c3c91ea78108" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "280ef0c29f8d4b3283f7c3c91ea78108" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44285 } } }
I20260618 22:32:11.556814 32528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "280ef0c29f8d4b3283f7c3c91ea78108" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "280ef0c29f8d4b3283f7c3c91ea78108" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44285 } } }
I20260618 22:32:11.557725 32529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:11.557775 32528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:11.558534 32533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:11.569829 32533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:11.579249 32533 catalog_manager.cc:1347] Generated new cluster ID: 357febe1a6f941609c3db6527fa87b43
I20260618 22:32:11.579380 32533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:11.600080 32533 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:11.600561 32533 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:11.610617 32533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 280ef0c29f8d4b3283f7c3c91ea78108: Generated new TSK 0
I20260618 22:32:11.611497 32533 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:32:11.625793 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 32450
W20260618 22:32:11.626926 32512 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.84:43465
W20260618 22:32:11.627604 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 32450: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:32:11.627725 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 32450
2026-06-18T22:32:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1806 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:11 dist-test-slave-0956 krb5kdc[32548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:11 dist-test-slave-0956 krb5kdc[32548](info): set up 2 sockets
Jun 18 22:32:11 dist-test-slave-0956 krb5kdc[32548](info): commencing operation
krb5kdc: starting...
W20260618 22:32:12.649243 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:32:12 dist-test-slave-0956 krb5kdc[32548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:32:12.900998 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39365
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:13.226790 32561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:13.227082 32561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:13.227201 32561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:13.234442 32561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:13.234555 32561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:13.234609 32561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:13.234692 32561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:13.234747 32561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:13.244625 32561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39365
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:13.245971 32561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:13.247244 32561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:13.258761 32567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:13.258898 32566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:13.258734 32569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:13.259548 32561 server_base.cc:1034] running on GCE node
I20260618 22:32:13.260295 32561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:13.261740 32561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:13.262959 32561 hybrid_clock.cc:648] HybridClock initialized: now 1781821933262891 us; error 54 us; skew 500 ppm
I20260618 22:32:13.263365 32561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:13 dist-test-slave-0956 krb5kdc[32548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821933, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:13.270386 32561 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:32:13.271878 32561 webserver.cc:460] Webserver started at http://127.0.0.1:33393/ using document root <none> and password file <none>
I20260618 22:32:13.272542 32561 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:13.272661 32561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:13.272954 32561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:13.275962 32561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "98eb2d8971f94d4e82360ffb538dee6d"
format_stamp: "Formatted at 2026-06-18 22:32:13 on dist-test-slave-0956"
server_key: "79e3ea0253f898afe8e2a831f899d857"
server_key_iv: "2427e5fa3495b463f152ac4fdb6ff2d6"
server_key_version: "encryptionkey@0"
I20260618 22:32:13.276698 32561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "98eb2d8971f94d4e82360ffb538dee6d"
format_stamp: "Formatted at 2026-06-18 22:32:13 on dist-test-slave-0956"
server_key: "79e3ea0253f898afe8e2a831f899d857"
server_key_iv: "2427e5fa3495b463f152ac4fdb6ff2d6"
server_key_version: "encryptionkey@0"
I20260618 22:32:13.281761 32561 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260618 22:32:13.285311 32576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:13.286626 32561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:32:13.286789 32561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "98eb2d8971f94d4e82360ffb538dee6d"
format_stamp: "Formatted at 2026-06-18 22:32:13 on dist-test-slave-0956"
server_key: "79e3ea0253f898afe8e2a831f899d857"
server_key_iv: "2427e5fa3495b463f152ac4fdb6ff2d6"
server_key_version: "encryptionkey@0"
I20260618 22:32:13.286981 32561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:13.305931 32561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:13.309736 32561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:13.309998 32561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:13.326848 32561 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39365
I20260618 22:32:13.326874 32627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39365 every 8 connection(s)
I20260618 22:32:13.328256 32561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:13.332088 32628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:13.334656 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 32561
I20260618 22:32:13.334808 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:32:13.335217 11211 external_mini_cluster.cc:1428] Setting key 53c9c02879d2b285c2c8821bd2b3f27d
I20260618 22:32:13.343750 32628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d: Bootstrap starting.
I20260618 22:32:13.347646 32628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:13.348954 32628 log.cc:824] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:32:13 dist-test-slave-0956 krb5kdc[32548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260618 22:32:13.351936 32628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d: No bootstrap required, opened a new log
I20260618 22:32:13.358134 32631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:13.336943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:53369 (local address 127.0.0.1:39365)
0618 22:32:13.337708 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:13.337721 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:13.337762 (+ 41us) server_negotiation.cc:407] Connection header received
0618 22:32:13.339412 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:13.339435 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:13.339894 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:13.340231 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:13.341927 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:13.343225 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:13.344941 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:13.345678 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:13.345814 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:13.350815 (+ 5001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:13.350823 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:13.350839 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:13.350875 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:13.353857 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:13.354731 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:13.354737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:13.354744 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:13.354912 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:13.355374 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:13.355378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:13.355380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:13.355921 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:13.356249 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:13.356737 (+ 488us) server_negotiation.cc:299] Negotiation successful
0618 22:32:13.357051 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":358,"threads_started":1}
I20260618 22:32:13.359570 32628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98eb2d8971f94d4e82360ffb538dee6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39365 } }
I20260618 22:32:13.359933 32628 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:13.360023 32628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98eb2d8971f94d4e82360ffb538dee6d, State: Initialized, Role: FOLLOWER
I20260618 22:32:13.360616 32628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98eb2d8971f94d4e82360ffb538dee6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39365 } }
I20260618 22:32:13.360884 32628 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:13.361013 32628 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:13.361227 32628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:13.361908 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:32:13.364235 32628 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98eb2d8971f94d4e82360ffb538dee6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39365 } }
I20260618 22:32:13.364841 32628 leader_election.cc:302] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98eb2d8971f94d4e82360ffb538dee6d; no voters:
I20260618 22:32:13.365563 32628 leader_election.cc:288] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:13.365686 32634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:13.366482 32634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d [term 1 LEADER]: Becoming Leader. State: Replica: 98eb2d8971f94d4e82360ffb538dee6d, State: Running, Role: LEADER
I20260618 22:32:13.367173 32634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98eb2d8971f94d4e82360ffb538dee6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39365 } }
I20260618 22:32:13.368963 32628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:13.375308 32639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98eb2d8971f94d4e82360ffb538dee6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98eb2d8971f94d4e82360ffb538dee6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39365 } } }
I20260618 22:32:13.375653 32639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:13.376076 32638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98eb2d8971f94d4e82360ffb538dee6d. Latest consensus state: current_term: 1 leader_uuid: "98eb2d8971f94d4e82360ffb538dee6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98eb2d8971f94d4e82360ffb538dee6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39365 } } }
I20260618 22:32:13.376380 32638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:13.377112 32652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:13.382455 32652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:13.385879 32631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:13.367440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:45745 (local address 127.0.0.1:39365)
0618 22:32:13.367615 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:13.367621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:13.368172 (+ 551us) server_negotiation.cc:407] Connection header received
0618 22:32:13.368350 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:13.368355 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:13.368448 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:13.368559 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:13.370666 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:13.371946 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:13.373716 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:13.374423 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:13.374563 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:13.377829 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:13.377836 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:13.377841 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:13.377884 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:13.381520 (+ 3636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:13.382628 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:13.382633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:13.382636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:13.382773 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:13.383378 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:13.383383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:13.383386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:13.383807 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:13.385130 (+ 1323us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:13.385313 (+ 183us) server_negotiation.cc:299] Negotiation successful
0618 22:32:13.385509 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:32:13.391848 32652 catalog_manager.cc:1347] Generated new cluster ID: 09a31bd25b6b4a52bdadd344e0a42381
I20260618 22:32:13.391974 32652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:13.410915 32652 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:13.412237 32652 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:13.425413 32652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98eb2d8971f94d4e82360ffb538dee6d: Generated new TSK 0
I20260618 22:32:13.426271 32652 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:32:13.432271 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 32561
I20260618 22:32:13.433511 32623 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:13.525159 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 32561
W20260618 22:32:13.538712 32546 connection.cc:535] client connection to 127.0.0.1:39365 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39365): Connection reset by peer (error 104)
2026-06-18T22:32:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1897 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:13 dist-test-slave-0956 krb5kdc[32660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:13 dist-test-slave-0956 krb5kdc[32660](info): set up 2 sockets
Jun 18 22:32:13 dist-test-slave-0956 krb5kdc[32660](info): commencing operation
krb5kdc: starting...
W20260618 22:32:14.541270 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:32:14 dist-test-slave-0956 krb5kdc[32660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:32:14.792706 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34931
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:15.124826 32673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:15.125186 32673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:15.125339 32673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:15.132886 32673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:15.133029 32673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:15.133178 32673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:15.133272 32673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:15.133333 32673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:15.143679 32673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34931
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:15.145061 32673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:15.146416 32673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:15.158627 32679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:15.158722 32678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:15.158702 32681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:15.159138 32673 server_base.cc:1034] running on GCE node
I20260618 22:32:15.159943 32673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:15.161458 32673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:15.162706 32673 hybrid_clock.cc:648] HybridClock initialized: now 1781821935162645 us; error 44 us; skew 500 ppm
I20260618 22:32:15.163153 32673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:15 dist-test-slave-0956 krb5kdc[32660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821935, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:15.166427 32673 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:32:15.168001 32673 webserver.cc:460] Webserver started at http://127.0.0.1:44289/ using document root <none> and password file <none>
I20260618 22:32:15.168700 32673 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:15.168833 32673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:15.169188 32673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:15.172394 32673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "72aa078e4e3c4d72966d3e69c7e89ce7"
format_stamp: "Formatted at 2026-06-18 22:32:15 on dist-test-slave-0956"
server_key: "2a936a19e7ab40acd37bfcd9a459c028"
server_key_iv: "54b6074514ec6d06e1cbcfd6e0704447"
server_key_version: "encryptionkey@0"
I20260618 22:32:15.173251 32673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "72aa078e4e3c4d72966d3e69c7e89ce7"
format_stamp: "Formatted at 2026-06-18 22:32:15 on dist-test-slave-0956"
server_key: "2a936a19e7ab40acd37bfcd9a459c028"
server_key_iv: "54b6074514ec6d06e1cbcfd6e0704447"
server_key_version: "encryptionkey@0"
I20260618 22:32:15.178968 32673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:32:15.183097 32688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:15.184773 32673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:32:15.184988 32673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "72aa078e4e3c4d72966d3e69c7e89ce7"
format_stamp: "Formatted at 2026-06-18 22:32:15 on dist-test-slave-0956"
server_key: "2a936a19e7ab40acd37bfcd9a459c028"
server_key_iv: "54b6074514ec6d06e1cbcfd6e0704447"
server_key_version: "encryptionkey@0"
I20260618 22:32:15.185263 32673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:15.209013 32673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:15.213492 32673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:15.213768 32673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:15.231475 32673 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34931
I20260618 22:32:15.231493 32739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34931 every 8 connection(s)
I20260618 22:32:15.233059 32673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:15.236837 32740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:15.237135 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 32673
I20260618 22:32:15.237313 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:32:15.237759 11211 external_mini_cluster.cc:1428] Setting key 00b94033cd816a86f951d6f38e73ea02
I20260618 22:32:15.250248 32740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7: Bootstrap starting.
I20260618 22:32:15.251965 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:15.239576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:56197 (local address 127.0.0.1:34931)
0618 22:32:15.240385 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:15.240397 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:15.240453 (+ 56us) server_negotiation.cc:407] Connection header received
0618 22:32:15.242654 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:15.242683 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:15.243195 (+ 512us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:15.243541 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:15.244140 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:15.245490 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:15.247380 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:15.248106 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:15.248248 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:15.249528 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:15.249535 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:15.249550 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:15.250124 (+ 574us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:15.250211 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:15.250240 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:15.250359 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:15.250691 (+ 332us) server_negotiation.cc:299] Negotiation successful
0618 22:32:15.250959 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":356,"threads_started":1}
I20260618 22:32:15.253583 32740 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:15.254683 32740 log.cc:824] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:15.255833 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:32:15.257689 32740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7: No bootstrap required, opened a new log
I20260618 22:32:15.266487 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:15.260032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:50681 (local address 127.0.0.1:34931)
0618 22:32:15.260217 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:15.260224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:15.260512 (+ 288us) server_negotiation.cc:407] Connection header received
0618 22:32:15.260661 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:15.260667 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:15.260752 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:15.260870 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:15.261386 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:15.262494 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:15.263842 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:15.264542 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:15.264673 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:15.265760 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:15.265767 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:15.265772 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:15.265794 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:15.265862 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:15.265882 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:15.265993 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:15.266117 (+ 124us) server_negotiation.cc:299] Negotiation successful
0618 22:32:15.266173 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:32:15.266778 32740 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72aa078e4e3c4d72966d3e69c7e89ce7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34931 } }
I20260618 22:32:15.267213 32740 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:15.267333 32740 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72aa078e4e3c4d72966d3e69c7e89ce7, State: Initialized, Role: FOLLOWER
I20260618 22:32:15.268010 32740 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72aa078e4e3c4d72966d3e69c7e89ce7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34931 } }
I20260618 22:32:15.268239 32740 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:15.268368 32740 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:15.268514 32740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:15.270874 32740 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72aa078e4e3c4d72966d3e69c7e89ce7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34931 } }
I20260618 22:32:15.271425 32740 leader_election.cc:302] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72aa078e4e3c4d72966d3e69c7e89ce7; no voters:
I20260618 22:32:15.272125 32740 leader_election.cc:288] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:15.272269 32750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:15.273517 32750 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7 [term 1 LEADER]: Becoming Leader. State: Replica: 72aa078e4e3c4d72966d3e69c7e89ce7, State: Running, Role: LEADER
I20260618 22:32:15.274219 32750 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72aa078e4e3c4d72966d3e69c7e89ce7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34931 } }
I20260618 22:32:15.274802 32740 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:15.276935 32751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72aa078e4e3c4d72966d3e69c7e89ce7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72aa078e4e3c4d72966d3e69c7e89ce7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34931 } } }
I20260618 22:32:15.277100 32752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72aa078e4e3c4d72966d3e69c7e89ce7. Latest consensus state: current_term: 1 leader_uuid: "72aa078e4e3c4d72966d3e69c7e89ce7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72aa078e4e3c4d72966d3e69c7e89ce7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34931 } } }
I20260618 22:32:15.277396 32751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:15.277395 32752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:15.279254 32759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:15.284544 32759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:15.293336 32759 catalog_manager.cc:1347] Generated new cluster ID: 676f93b23cbe4e4aae0e47ddf1b4777d
I20260618 22:32:15.293460 32759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:15.307112 32759 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:15.308941 32759 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:15.317400 32759 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72aa078e4e3c4d72966d3e69c7e89ce7: Generated new TSK 0
I20260618 22:32:15.318473 32759 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:32:15.335327 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 32673
W20260618 22:32:15.336440 32735 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.84:56197
W20260618 22:32:15.337160 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 32673: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:32:15.337282 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 32673
W20260618 22:32:15.350950 32658 connection.cc:535] client connection to 127.0.0.1:34931 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34931): Connection reset by peer (error 104)
2026-06-18T22:32:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1811 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:15 dist-test-slave-0956 krb5kdc[303](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:15 dist-test-slave-0956 krb5kdc[303](info): set up 2 sockets
Jun 18 22:32:15 dist-test-slave-0956 krb5kdc[303](info): commencing operation
krb5kdc: starting...
W20260618 22:32:16.349327 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:32:16 dist-test-slave-0956 krb5kdc[303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:32:16.592257 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42833
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:16.915606 316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:16.915915 316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:16.916019 316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:16.923312 316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:16.923429 316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:16.923483 316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:16.923542 316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:16.923585 316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:16.933521 316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42833
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:16.934795 316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:16.936064 316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:16.947674 322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:16.947750 321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:16.947708 324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:16.948719 316 server_base.cc:1034] running on GCE node
I20260618 22:32:16.949609 316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:16.951135 316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:16.952368 316 hybrid_clock.cc:648] HybridClock initialized: now 1781821936952322 us; error 40 us; skew 500 ppm
I20260618 22:32:16.952741 316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:16 dist-test-slave-0956 krb5kdc[303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821936, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:16.955955 316 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:32:16.957515 316 webserver.cc:460] Webserver started at http://127.0.0.1:45359/ using document root <none> and password file <none>
I20260618 22:32:16.958225 316 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:16.958340 316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:16.958627 316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:16.961637 316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "76b1745ef7604e2aafe2a19d74aebf92"
format_stamp: "Formatted at 2026-06-18 22:32:16 on dist-test-slave-0956"
server_key: "ea9cb920a8b37e79f55a631bcd58dcf2"
server_key_iv: "72b1a3b5114e34f7b60d9482f7bd651e"
server_key_version: "encryptionkey@0"
I20260618 22:32:16.962402 316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "76b1745ef7604e2aafe2a19d74aebf92"
format_stamp: "Formatted at 2026-06-18 22:32:16 on dist-test-slave-0956"
server_key: "ea9cb920a8b37e79f55a631bcd58dcf2"
server_key_iv: "72b1a3b5114e34f7b60d9482f7bd651e"
server_key_version: "encryptionkey@0"
I20260618 22:32:16.967511 316 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:32:16.970983 331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:16.972323 316 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:32:16.972482 316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "76b1745ef7604e2aafe2a19d74aebf92"
format_stamp: "Formatted at 2026-06-18 22:32:16 on dist-test-slave-0956"
server_key: "ea9cb920a8b37e79f55a631bcd58dcf2"
server_key_iv: "72b1a3b5114e34f7b60d9482f7bd651e"
server_key_version: "encryptionkey@0"
I20260618 22:32:16.972679 316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:16.978526 316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:16.983371 316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:16.983680 316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:17.002647 316 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42833
I20260618 22:32:17.002676 382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42833 every 8 connection(s)
I20260618 22:32:17.004156 316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:17.005827 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 316
I20260618 22:32:17.006007 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:32:17.006518 11211 external_mini_cluster.cc:1428] Setting key c0b6930a82995453df704931e772f6d8
I20260618 22:32:17.008580 383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:17.019951 386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:17.008493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:56135 (local address 127.0.0.1:42833)
0618 22:32:17.009381 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:17.009396 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:17.013688 (+ 4292us) server_negotiation.cc:407] Connection header received
0618 22:32:17.015978 (+ 2290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:17.016007 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:17.016475 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:17.016932 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:17.017969 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:17.017978 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:17.017996 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:17.018017 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:17.018093 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:17.018125 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:17.018348 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:17.018480 (+ 132us) server_negotiation.cc:299] Negotiation successful
0618 22:32:17.018689 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":339,"threads_started":1}
I20260618 22:32:17.020784 383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92: Bootstrap starting.
I20260618 22:32:17.023488 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:32:17.024063 383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:17.025291 383 log.cc:824] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:17.028506 383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92: No bootstrap required, opened a new log
I20260618 22:32:17.030833 386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:17.028334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:36879 (local address 127.0.0.1:42833)
0618 22:32:17.028503 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:17.028509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:17.029015 (+ 506us) server_negotiation.cc:407] Connection header received
0618 22:32:17.029203 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:17.029207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:17.029255 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:17.029330 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:17.030221 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:17.030225 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:17.030229 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:17.030243 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:17.030283 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:17.030301 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:17.030384 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:17.030504 (+ 120us) server_negotiation.cc:299] Negotiation successful
0618 22:32:17.030559 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260618 22:32:17.036525 383 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76b1745ef7604e2aafe2a19d74aebf92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42833 } }
I20260618 22:32:17.037002 383 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:17.037120 383 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76b1745ef7604e2aafe2a19d74aebf92, State: Initialized, Role: FOLLOWER
I20260618 22:32:17.037933 383 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76b1745ef7604e2aafe2a19d74aebf92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42833 } }
I20260618 22:32:17.038206 383 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:17.038345 383 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:17.038529 383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:17.041435 383 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76b1745ef7604e2aafe2a19d74aebf92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42833 } }
I20260618 22:32:17.041926 383 leader_election.cc:302] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76b1745ef7604e2aafe2a19d74aebf92; no voters:
I20260618 22:32:17.042438 383 leader_election.cc:288] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:17.042577 393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:17.043166 393 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92 [term 1 LEADER]: Becoming Leader. State: Replica: 76b1745ef7604e2aafe2a19d74aebf92, State: Running, Role: LEADER
I20260618 22:32:17.043747 393 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76b1745ef7604e2aafe2a19d74aebf92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42833 } }
I20260618 22:32:17.044461 383 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:17.046236 395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76b1745ef7604e2aafe2a19d74aebf92. Latest consensus state: current_term: 1 leader_uuid: "76b1745ef7604e2aafe2a19d74aebf92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76b1745ef7604e2aafe2a19d74aebf92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42833 } } }
I20260618 22:32:17.046466 394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76b1745ef7604e2aafe2a19d74aebf92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76b1745ef7604e2aafe2a19d74aebf92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42833 } } }
I20260618 22:32:17.046674 395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:17.046710 394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:17.049723 398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:17.055727 398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:17.065404 398 catalog_manager.cc:1347] Generated new cluster ID: 6bd2c310f65c42a2aaebcfc2e25c2593
I20260618 22:32:17.065524 398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:17.107154 398 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:17.107717 398 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:17.118642 398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76b1745ef7604e2aafe2a19d74aebf92: Generated new TSK 0
I20260618 22:32:17.119536 398 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:32:17.125990 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 316
W20260618 22:32:17.127053 378 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.84:56135
W20260618 22:32:17.127772 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 316: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:32:17.127924 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 316
2026-06-18T22:32:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1791 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:17 dist-test-slave-0956 krb5kdc[414](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:17 dist-test-slave-0956 krb5kdc[414](info): set up 2 sockets
Jun 18 22:32:17 dist-test-slave-0956 krb5kdc[414](info): commencing operation
krb5kdc: starting...
W20260618 22:32:18.145620 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.005s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:32:18 dist-test-slave-0956 krb5kdc[414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:32:18.408792 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:43711
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:40213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:43711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:18.741917 427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:18.742244 427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:18.742374 427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:18.749927 427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:18.750058 427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:18.750115 427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:18.750201 427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:18.750255 427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:18.760277 427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:43711
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:43711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:18.761674 427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:18.763028 427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:18.775058 435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:18.775177 432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:18.775147 433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:18.776475 427 server_base.cc:1034] running on GCE node
I20260618 22:32:18.777396 427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:18.778915 427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:18.780148 427 hybrid_clock.cc:648] HybridClock initialized: now 1781821938780102 us; error 36 us; skew 500 ppm
I20260618 22:32:18.780498 427 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:18 dist-test-slave-0956 krb5kdc[414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821938, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:18.787899 427 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:32:18.789594 427 webserver.cc:460] Webserver started at http://127.10.242.254:38637/ using document root <none> and password file <none>
I20260618 22:32:18.790310 427 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:18.790429 427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:18.790721 427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:18.793884 427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "dcd40bcbde3b4592bfc87414cc3ca4b3"
format_stamp: "Formatted at 2026-06-18 22:32:18 on dist-test-slave-0956"
server_key: "4657cb0b16059c519908919176627a75"
server_key_iv: "8974dd77268cda4413b44c1d6b87bc83"
server_key_version: "encryptionkey@0"
I20260618 22:32:18.794691 427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "dcd40bcbde3b4592bfc87414cc3ca4b3"
format_stamp: "Formatted at 2026-06-18 22:32:18 on dist-test-slave-0956"
server_key: "4657cb0b16059c519908919176627a75"
server_key_iv: "8974dd77268cda4413b44c1d6b87bc83"
server_key_version: "encryptionkey@0"
I20260618 22:32:18.800189 427 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:32:18.804093 442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:18.805635 427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:32:18.805810 427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "dcd40bcbde3b4592bfc87414cc3ca4b3"
format_stamp: "Formatted at 2026-06-18 22:32:18 on dist-test-slave-0956"
server_key: "4657cb0b16059c519908919176627a75"
server_key_iv: "8974dd77268cda4413b44c1d6b87bc83"
server_key_version: "encryptionkey@0"
I20260618 22:32:18.806005 427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:18.818611 427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:18.822851 427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:18.823130 427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:18.840340 427 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:43711
I20260618 22:32:18.840348 493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:43711 every 8 connection(s)
I20260618 22:32:18.841826 427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:18.843039 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 427
I20260618 22:32:18.843264 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:32:18.843792 11211 external_mini_cluster.cc:1428] Setting key 6c7de1213c2fb67bb322bbbb5c48505f
I20260618 22:32:18.846658 494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:18.862828 494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3: Bootstrap starting.
Jun 18 22:32:18 dist-test-slave-0956 krb5kdc[414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:32:18.866473 494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:18.867686 494 log.cc:824] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:18.871049 494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3: No bootstrap required, opened a new log
I20260618 22:32:18.873502 497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:18.845641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37545 (local address 127.10.242.254:43711)
0618 22:32:18.851439 (+ 5798us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:18.851457 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:18.851511 (+ 54us) server_negotiation.cc:407] Connection header received
0618 22:32:18.853907 (+ 2396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:18.853939 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:18.854427 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:18.854895 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:18.856334 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:18.857984 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:18.859688 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:18.860794 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:18.860983 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:18.865322 (+ 4339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:18.865331 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:18.865345 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:18.865385 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:18.868990 (+ 3605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:18.869993 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:18.869999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:18.870006 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:18.870219 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:18.870719 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:18.870722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:18.870724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:18.871309 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:18.871549 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:18.872034 (+ 485us) server_negotiation.cc:299] Negotiation successful
0618 22:32:18.872345 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5531,"thread_start_us":5150,"threads_started":1}
I20260618 22:32:18.877380 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:32:18.880038 494 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcd40bcbde3b4592bfc87414cc3ca4b3" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43711 } }
I20260618 22:32:18.880540 494 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:18.880668 494 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcd40bcbde3b4592bfc87414cc3ca4b3, State: Initialized, Role: FOLLOWER
I20260618 22:32:18.881400 494 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcd40bcbde3b4592bfc87414cc3ca4b3" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43711 } }
I20260618 22:32:18.881686 494 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:18.881829 494 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:18.881999 494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:18.885171 494 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcd40bcbde3b4592bfc87414cc3ca4b3" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43711 } }
I20260618 22:32:18.885782 494 leader_election.cc:302] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcd40bcbde3b4592bfc87414cc3ca4b3; no voters:
I20260618 22:32:18.886384 494 leader_election.cc:288] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:18.886502 504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:18.888077 504 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3 [term 1 LEADER]: Becoming Leader. State: Replica: dcd40bcbde3b4592bfc87414cc3ca4b3, State: Running, Role: LEADER
I20260618 22:32:18.888823 494 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:18.888747 504 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcd40bcbde3b4592bfc87414cc3ca4b3" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43711 } }
I20260618 22:32:18.897547 497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:18.882665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56111 (local address 127.10.242.254:43711)
0618 22:32:18.882857 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:18.882867 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:18.883383 (+ 516us) server_negotiation.cc:407] Connection header received
0618 22:32:18.883548 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:18.883551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:18.883618 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:18.883705 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:18.885458 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:18.886289 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:18.887868 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:18.888487 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:18.888598 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:18.891341 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:18.891348 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:18.891353 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:18.891393 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:18.894093 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:18.894944 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:18.894949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:18.894951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:18.895079 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:18.895515 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:18.895519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:18.895521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:18.895850 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:18.896112 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:18.896704 (+ 592us) server_negotiation.cc:299] Negotiation successful
0618 22:32:18.897107 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:32:18.898713 505 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcd40bcbde3b4592bfc87414cc3ca4b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcd40bcbde3b4592bfc87414cc3ca4b3" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43711 } } }
I20260618 22:32:18.898785 506 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcd40bcbde3b4592bfc87414cc3ca4b3. Latest consensus state: current_term: 1 leader_uuid: "dcd40bcbde3b4592bfc87414cc3ca4b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcd40bcbde3b4592bfc87414cc3ca4b3" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43711 } } }
I20260618 22:32:18.899224 505 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:18.899243 506 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:18.903110 514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:18.908352 514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:18.917781 514 catalog_manager.cc:1347] Generated new cluster ID: 64b69a3d33274c8692a8cb9b38e11c3c
I20260618 22:32:18.917913 514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:18.927649 514 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:18.929523 514 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:18.940321 514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcd40bcbde3b4592bfc87414cc3ca4b3: Generated new TSK 0
I20260618 22:32:18.941542 514 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:32:18.948956 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 427
I20260618 22:32:18.950258 489 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:19.040427 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 427
W20260618 22:32:19.055862 412 connection.cc:535] client connection to 127.10.242.254:43711 recv error: Network error: failed to read from TLS socket (remote: 127.10.242.254:43711): Connection reset by peer (error 104)
2026-06-18T22:32:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1913 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:19 dist-test-slave-0956 krb5kdc[526](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:19 dist-test-slave-0956 krb5kdc[526](info): set up 2 sockets
Jun 18 22:32:19 dist-test-slave-0956 krb5kdc[526](info): commencing operation
krb5kdc: starting...
W20260618 22:32:20.077389 11211 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.006s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:32:20 dist-test-slave-0956 krb5kdc[526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:32:20.321622 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:45183
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:37597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:45183
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:20.647373 539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:20.647696 539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:20.647840 539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:20.655257 539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:20.655404 539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:20.655493 539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:20.655573 539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:20.655633 539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:20.665831 539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:37597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:45183
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:45183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:20.667194 539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:20.668499 539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:20.679862 544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:20.679970 545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:20.679977 547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:20.681035 539 server_base.cc:1034] running on GCE node
I20260618 22:32:20.681895 539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:20.683334 539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:20.684530 539 hybrid_clock.cc:648] HybridClock initialized: now 1781821940684478 us; error 48 us; skew 500 ppm
I20260618 22:32:20.684881 539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:20 dist-test-slave-0956 krb5kdc[526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821940, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:20.687978 539 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:32:20.689463 539 webserver.cc:460] Webserver started at http://127.10.242.254:37393/ using document root <none> and password file <none>
I20260618 22:32:20.690111 539 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:20.690210 539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:20.690448 539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:20.693380 539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "e8e1eed90fa84c18a02bd323de207c59"
format_stamp: "Formatted at 2026-06-18 22:32:20 on dist-test-slave-0956"
server_key: "7c86e8142306b46b884bdc3b10652c8a"
server_key_iv: "30d0a58c086a08bb9f28feb1c9e4676b"
server_key_version: "encryptionkey@0"
I20260618 22:32:20.694161 539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "e8e1eed90fa84c18a02bd323de207c59"
format_stamp: "Formatted at 2026-06-18 22:32:20 on dist-test-slave-0956"
server_key: "7c86e8142306b46b884bdc3b10652c8a"
server_key_iv: "30d0a58c086a08bb9f28feb1c9e4676b"
server_key_version: "encryptionkey@0"
I20260618 22:32:20.699335 539 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:32:20.702797 554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:20.704288 539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:32:20.704448 539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "e8e1eed90fa84c18a02bd323de207c59"
format_stamp: "Formatted at 2026-06-18 22:32:20 on dist-test-slave-0956"
server_key: "7c86e8142306b46b884bdc3b10652c8a"
server_key_iv: "30d0a58c086a08bb9f28feb1c9e4676b"
server_key_version: "encryptionkey@0"
I20260618 22:32:20.704643 539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:20.729707 539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:20.734205 539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:20.734488 539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:20.752476 539 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:45183
I20260618 22:32:20.752504 605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:45183 every 8 connection(s)
I20260618 22:32:20.753901 539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:20.755210 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 539
I20260618 22:32:20.755385 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:32:20.755918 11211 external_mini_cluster.cc:1428] Setting key 56acc23e092c9e41a261f6113a4f06a0
I20260618 22:32:20.758517 606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:20.769518 609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:20.757760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58153 (local address 127.10.242.254:45183)
0618 22:32:20.758512 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:20.758525 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:20.758569 (+ 44us) server_negotiation.cc:407] Connection header received
0618 22:32:20.760176 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:20.760198 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:20.760587 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:20.760926 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:20.761531 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:20.762915 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:20.764739 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:20.765566 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:20.765709 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:20.766994 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:20.767002 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:20.767018 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:20.767600 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:20.767702 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:20.767736 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:20.767872 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:20.768199 (+ 327us) server_negotiation.cc:299] Negotiation successful
0618 22:32:20.768407 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":335,"threads_started":1}
I20260618 22:32:20.770823 606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59: Bootstrap starting.
I20260618 22:32:20.773392 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:32:20.774299 606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:20.775607 606 log.cc:824] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:20.779276 606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59: No bootstrap required, opened a new log
I20260618 22:32:20.785923 609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:20.778642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51367 (local address 127.10.242.254:45183)
0618 22:32:20.778786 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:20.778791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:20.779169 (+ 378us) server_negotiation.cc:407] Connection header received
0618 22:32:20.779405 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:20.779411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:20.779482 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:20.779569 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:20.780235 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:20.781035 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:20.782948 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:20.783472 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:20.783581 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:20.784988 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:20.784996 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:20.784999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:20.785027 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:20.785070 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:20.785156 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:20.785254 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:20.785516 (+ 262us) server_negotiation.cc:299] Negotiation successful
0618 22:32:20.785582 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260618 22:32:20.788993 606 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e1eed90fa84c18a02bd323de207c59" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45183 } }
I20260618 22:32:20.789456 606 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:20.789561 606 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8e1eed90fa84c18a02bd323de207c59, State: Initialized, Role: FOLLOWER
I20260618 22:32:20.790149 606 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e1eed90fa84c18a02bd323de207c59" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45183 } }
I20260618 22:32:20.790360 606 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:20.790495 606 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:20.790659 606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:20.792902 606 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e1eed90fa84c18a02bd323de207c59" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45183 } }
I20260618 22:32:20.793565 606 leader_election.cc:302] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8e1eed90fa84c18a02bd323de207c59; no voters:
I20260618 22:32:20.794301 606 leader_election.cc:288] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:20.794405 616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:20.795029 616 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59 [term 1 LEADER]: Becoming Leader. State: Replica: e8e1eed90fa84c18a02bd323de207c59, State: Running, Role: LEADER
I20260618 22:32:20.795626 616 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e1eed90fa84c18a02bd323de207c59" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45183 } }
I20260618 22:32:20.796401 606 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:20.798651 617 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8e1eed90fa84c18a02bd323de207c59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e1eed90fa84c18a02bd323de207c59" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45183 } } }
I20260618 22:32:20.798583 618 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8e1eed90fa84c18a02bd323de207c59. Latest consensus state: current_term: 1 leader_uuid: "e8e1eed90fa84c18a02bd323de207c59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e1eed90fa84c18a02bd323de207c59" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 45183 } } }
I20260618 22:32:20.799125 617 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:20.799150 618 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:20.800595 622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:20.806950 622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:20.817656 622 catalog_manager.cc:1347] Generated new cluster ID: b92474954e1f4349bd2f86d7f4431c38
I20260618 22:32:20.817780 622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:20.831053 622 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:20.832906 622 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:20.843515 622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8e1eed90fa84c18a02bd323de207c59: Generated new TSK 0
I20260618 22:32:20.844647 622 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:32:20.856501 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 539
W20260618 22:32:20.857795 601 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58153
W20260618 22:32:20.858551 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 539: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:32:20.858690 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 539
W20260618 22:32:20.872066 524 connection.cc:535] client connection to 127.10.242.254:45183 recv error: Network error: failed to read from TLS socket (remote: 127.10.242.254:45183): Connection reset by peer (error 104)
2026-06-18T22:32:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1816 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:20 dist-test-slave-0956 krb5kdc[637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:20 dist-test-slave-0956 krb5kdc[637](info): set up 2 sockets
Jun 18 22:32:20 dist-test-slave-0956 krb5kdc[637](info): commencing operation
krb5kdc: starting...
W20260618 22:32:21.885437 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:32:22 dist-test-slave-0956 krb5kdc[637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:32:22.132301 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:40605
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:44349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:40605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:22.459256 650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:22.459568 650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:22.459702 650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:22.467000 650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:22.467139 650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:22.467219 650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:22.467298 650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:22.467355 650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:22.477752 650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:44349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:40605
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:40605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:22.479022 650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:22.480335 650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:22.492043 658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:22.492134 656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:22.492082 655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:22.492763 650 server_base.cc:1034] running on GCE node
I20260618 22:32:22.493579 650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:22.495057 650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:22.496366 650 hybrid_clock.cc:648] HybridClock initialized: now 1781821942496238 us; error 46 us; skew 500 ppm
I20260618 22:32:22.496750 650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:22 dist-test-slave-0956 krb5kdc[637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821942, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:22.499830 650 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:32:22.501292 650 webserver.cc:460] Webserver started at http://127.10.242.254:33311/ using document root <none> and password file <none>
I20260618 22:32:22.501933 650 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:22.502040 650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:22.502286 650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:22.505210 650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "ac9e1d5b23944dbab90ee033151a7fbc"
format_stamp: "Formatted at 2026-06-18 22:32:22 on dist-test-slave-0956"
server_key: "8d63145e70cfba7986e2600a4384911b"
server_key_iv: "dab266a6578bad07d960250873c7f535"
server_key_version: "encryptionkey@0"
I20260618 22:32:22.505901 650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "ac9e1d5b23944dbab90ee033151a7fbc"
format_stamp: "Formatted at 2026-06-18 22:32:22 on dist-test-slave-0956"
server_key: "8d63145e70cfba7986e2600a4384911b"
server_key_iv: "dab266a6578bad07d960250873c7f535"
server_key_version: "encryptionkey@0"
I20260618 22:32:22.510766 650 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:32:22.514343 665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:22.515708 650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:32:22.515877 650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "ac9e1d5b23944dbab90ee033151a7fbc"
format_stamp: "Formatted at 2026-06-18 22:32:22 on dist-test-slave-0956"
server_key: "8d63145e70cfba7986e2600a4384911b"
server_key_iv: "dab266a6578bad07d960250873c7f535"
server_key_version: "encryptionkey@0"
I20260618 22:32:22.516068 650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:22.521886 650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:22.526198 650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:22.526463 650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:22.543421 650 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:40605
I20260618 22:32:22.543449 716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:40605 every 8 connection(s)
I20260618 22:32:22.544862 650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:22.546042 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 650
I20260618 22:32:22.546224 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:32:22.546761 11211 external_mini_cluster.cc:1428] Setting key a7493e745ae59053acc84a2069aebb31
I20260618 22:32:22.549674 717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:22.554878 720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:22.548641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33297 (local address 127.10.242.254:40605)
0618 22:32:22.549531 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:22.549544 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:22.549586 (+ 42us) server_negotiation.cc:407] Connection header received
0618 22:32:22.551299 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:22.551322 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:22.551682 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:22.552099 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:22.553165 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:22.553172 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:22.553190 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:22.553206 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:22.553288 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:22.553311 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:22.553516 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:22.553676 (+ 160us) server_negotiation.cc:299] Negotiation successful
0618 22:32:22.553879 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":446,"threads_started":1}
I20260618 22:32:22.558856 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:32:22.561280 717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc: Bootstrap starting.
I20260618 22:32:22.564631 717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:22.565781 720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:22.563119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36719 (local address 127.10.242.254:40605)
0618 22:32:22.563327 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:22.563334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:22.563811 (+ 477us) server_negotiation.cc:407] Connection header received
0618 22:32:22.563980 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:22.563984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:22.564035 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:22.564117 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:22.565138 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:22.565143 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:22.565146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:22.565163 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:22.565211 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:22.565227 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:22.565329 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:22.565453 (+ 124us) server_negotiation.cc:299] Negotiation successful
0618 22:32:22.565522 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:32:22.565850 717 log.cc:824] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:22.568883 717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc: No bootstrap required, opened a new log
I20260618 22:32:22.576371 717 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac9e1d5b23944dbab90ee033151a7fbc" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40605 } }
I20260618 22:32:22.576749 717 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:22.576849 717 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac9e1d5b23944dbab90ee033151a7fbc, State: Initialized, Role: FOLLOWER
I20260618 22:32:22.577525 717 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac9e1d5b23944dbab90ee033151a7fbc" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40605 } }
I20260618 22:32:22.577818 717 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:22.577957 717 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:22.578116 717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:22.580230 717 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac9e1d5b23944dbab90ee033151a7fbc" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40605 } }
I20260618 22:32:22.580717 717 leader_election.cc:302] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac9e1d5b23944dbab90ee033151a7fbc; no voters:
I20260618 22:32:22.581322 717 leader_election.cc:288] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:22.581435 727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:22.582091 727 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc [term 1 LEADER]: Becoming Leader. State: Replica: ac9e1d5b23944dbab90ee033151a7fbc, State: Running, Role: LEADER
I20260618 22:32:22.582633 727 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac9e1d5b23944dbab90ee033151a7fbc" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40605 } }
I20260618 22:32:22.583267 717 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:22.585969 728 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac9e1d5b23944dbab90ee033151a7fbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac9e1d5b23944dbab90ee033151a7fbc" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40605 } } }
I20260618 22:32:22.586412 728 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:22.585978 729 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac9e1d5b23944dbab90ee033151a7fbc. Latest consensus state: current_term: 1 leader_uuid: "ac9e1d5b23944dbab90ee033151a7fbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac9e1d5b23944dbab90ee033151a7fbc" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 40605 } } }
I20260618 22:32:22.586788 729 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:22.587623 736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:22.593402 736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:22.602068 736 catalog_manager.cc:1347] Generated new cluster ID: 625eebca9f444e37b36f2bb91a892cc7
I20260618 22:32:22.602191 736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:22.622347 736 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:22.622891 736 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:22.639331 736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac9e1d5b23944dbab90ee033151a7fbc: Generated new TSK 0
I20260618 22:32:22.640241 736 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:32:22.651607 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 650
W20260618 22:32:22.652755 712 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33297
W20260618 22:32:22.653540 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 650: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:32:22.653681 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 650
2026-06-18T22:32:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1795 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:22 dist-test-slave-0956 krb5kdc[748](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:22 dist-test-slave-0956 krb5kdc[748](info): set up 2 sockets
Jun 18 22:32:22 dist-test-slave-0956 krb5kdc[748](info): commencing operation
krb5kdc: starting...
W20260618 22:32:23.673414 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:32:23 dist-test-slave-0956 krb5kdc[748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:32:23.924078 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:39487
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:37529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:39487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:24.243323 761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:24.243697 761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:24.243854 761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:24.251332 761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:24.251463 761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:24.251523 761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:24.251606 761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:24.251664 761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:24.261780 761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:37529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:39487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:39487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:24.263068 761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:24.264364 761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:24.275599 767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:24.275674 766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:24.275651 769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:24.276214 761 server_base.cc:1034] running on GCE node
I20260618 22:32:24.276927 761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:24.278388 761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:24.279598 761 hybrid_clock.cc:648] HybridClock initialized: now 1781821944279563 us; error 59 us; skew 500 ppm
I20260618 22:32:24.279973 761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:24 dist-test-slave-0956 krb5kdc[748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821944, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:24.287892 761 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:32:24.289644 761 webserver.cc:460] Webserver started at http://127.10.242.254:39095/ using document root <none> and password file <none>
I20260618 22:32:24.290365 761 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:24.290488 761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:24.290786 761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:24.293891 761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "03c3919230154f65b7a5f3759e9101dc"
format_stamp: "Formatted at 2026-06-18 22:32:24 on dist-test-slave-0956"
server_key: "5c295d47bd78e2aea76d093811c2c68a"
server_key_iv: "239a7b995aee158de6fef2ba2134521a"
server_key_version: "encryptionkey@0"
I20260618 22:32:24.294621 761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "03c3919230154f65b7a5f3759e9101dc"
format_stamp: "Formatted at 2026-06-18 22:32:24 on dist-test-slave-0956"
server_key: "5c295d47bd78e2aea76d093811c2c68a"
server_key_iv: "239a7b995aee158de6fef2ba2134521a"
server_key_version: "encryptionkey@0"
I20260618 22:32:24.299919 761 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:32:24.303682 776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:24.305253 761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:32:24.305433 761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "03c3919230154f65b7a5f3759e9101dc"
format_stamp: "Formatted at 2026-06-18 22:32:24 on dist-test-slave-0956"
server_key: "5c295d47bd78e2aea76d093811c2c68a"
server_key_iv: "239a7b995aee158de6fef2ba2134521a"
server_key_version: "encryptionkey@0"
I20260618 22:32:24.305624 761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:24.332043 761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:24.336349 761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:24.336629 761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:24.354382 761 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:39487
I20260618 22:32:24.354393 827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:39487 every 8 connection(s)
I20260618 22:32:24.355759 761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:24.357499 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 761
I20260618 22:32:24.357677 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:32:24.358259 11211 external_mini_cluster.cc:1428] Setting key 7603776d9752c8848d4723123be8eca0
I20260618 22:32:24.360473 828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 18 22:32:24 dist-test-slave-0956 krb5kdc[748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:32:24.373075 828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc: Bootstrap starting.
I20260618 22:32:24.377241 828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:24.378427 828 log.cc:824] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:24.381599 831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:24.360040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33955 (local address 127.10.242.254:39487)
0618 22:32:24.360802 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:24.360813 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:24.360857 (+ 44us) server_negotiation.cc:407] Connection header received
0618 22:32:24.362499 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:24.362520 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:24.362908 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:24.363246 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:24.364911 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:24.366283 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:24.367951 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:24.368614 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:24.368736 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:24.373281 (+ 4545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:24.373293 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:24.373312 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:24.373355 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:24.377012 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:24.377724 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:24.377746 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:24.377754 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:24.377956 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:24.378316 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:24.378321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:24.378324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:24.379064 (+ 740us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:24.379316 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:24.379753 (+ 437us) server_negotiation.cc:299] Negotiation successful
0618 22:32:24.380273 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":352,"threads_started":1}
I20260618 22:32:24.381893 828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc: No bootstrap required, opened a new log
I20260618 22:32:24.385380 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:32:24.391516 828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c3919230154f65b7a5f3759e9101dc" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39487 } }
I20260618 22:32:24.392035 828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:24.392140 828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03c3919230154f65b7a5f3759e9101dc, State: Initialized, Role: FOLLOWER
I20260618 22:32:24.392834 828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c3919230154f65b7a5f3759e9101dc" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39487 } }
I20260618 22:32:24.393143 828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:24.393306 828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:24.393471 828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:24.396373 828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c3919230154f65b7a5f3759e9101dc" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39487 } }
I20260618 22:32:24.396981 828 leader_election.cc:302] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03c3919230154f65b7a5f3759e9101dc; no voters:
I20260618 22:32:24.397665 828 leader_election.cc:288] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:24.397778 838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:24.399252 838 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc [term 1 LEADER]: Becoming Leader. State: Replica: 03c3919230154f65b7a5f3759e9101dc, State: Running, Role: LEADER
I20260618 22:32:24.399880 828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:24.399904 838 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c3919230154f65b7a5f3759e9101dc" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39487 } }
I20260618 22:32:24.402740 840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03c3919230154f65b7a5f3759e9101dc. Latest consensus state: current_term: 1 leader_uuid: "03c3919230154f65b7a5f3759e9101dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c3919230154f65b7a5f3759e9101dc" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39487 } } }
I20260618 22:32:24.403033 840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:24.403534 839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03c3919230154f65b7a5f3759e9101dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c3919230154f65b7a5f3759e9101dc" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 39487 } } }
I20260618 22:32:24.403818 839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:24.403904 843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:24.407327 831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:24.390364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53675 (local address 127.10.242.254:39487)
0618 22:32:24.390565 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:24.390571 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:24.391076 (+ 505us) server_negotiation.cc:407] Connection header received
0618 22:32:24.391246 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:24.391255 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:24.391316 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:24.391391 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:24.393251 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:24.394052 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:24.395697 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:24.396127 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:24.396223 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:24.399094 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:24.399102 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:24.399108 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:24.399152 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:24.402846 (+ 3694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:24.403791 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:24.403796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:24.403800 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:24.403981 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:24.404507 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:24.404513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:24.404516 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:24.404939 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:24.405260 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:24.406645 (+ 1385us) server_negotiation.cc:299] Negotiation successful
0618 22:32:24.406896 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:32:24.411520 843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:24.421144 843 catalog_manager.cc:1347] Generated new cluster ID: 5ba7d0c4e35641289dc9a4fdc1206ffa
I20260618 22:32:24.421268 843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:24.429373 843 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:24.430894 843 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:24.443107 843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03c3919230154f65b7a5f3759e9101dc: Generated new TSK 0
I20260618 22:32:24.444058 843 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:32:24.454172 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 761
I20260618 22:32:24.455329 823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:24.544631 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 761
2026-06-18T22:32:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1890 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:24 dist-test-slave-0956 krb5kdc[860](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:24 dist-test-slave-0956 krb5kdc[860](info): set up 2 sockets
Jun 18 22:32:24 dist-test-slave-0956 krb5kdc[860](info): commencing operation
krb5kdc: starting...
W20260618 22:32:25.561372 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.005s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:32:25 dist-test-slave-0956 krb5kdc[860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:32:25.814340 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:44571
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:34147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:44571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:26.144183 873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:26.144488 873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:26.144608 873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:26.151978 873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:26.152117 873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:26.152164 873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:26.152230 873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:26.152292 873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:26.162933 873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:34147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:44571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:44571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:26.164306 873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:26.165721 873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:26.177887 878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:26.177865 879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:26.178069 881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:26.178984 873 server_base.cc:1034] running on GCE node
I20260618 22:32:26.179771 873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:26.181288 873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:26.182569 873 hybrid_clock.cc:648] HybridClock initialized: now 1781821946182516 us; error 45 us; skew 500 ppm
I20260618 22:32:26.182969 873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:26 dist-test-slave-0956 krb5kdc[860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821946, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:26.186455 873 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:32:26.188263 873 webserver.cc:460] Webserver started at http://127.10.242.254:46591/ using document root <none> and password file <none>
I20260618 22:32:26.188994 873 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:26.189162 873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:26.189769 873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:26.193610 873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "3e1bc07964014ec3ab8d453fb199e2fb"
format_stamp: "Formatted at 2026-06-18 22:32:26 on dist-test-slave-0956"
server_key: "74dd00c537bab0434a7ae6c3791288fc"
server_key_iv: "ec2f15530b79996e484566cbb54339c5"
server_key_version: "encryptionkey@0"
I20260618 22:32:26.194487 873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "3e1bc07964014ec3ab8d453fb199e2fb"
format_stamp: "Formatted at 2026-06-18 22:32:26 on dist-test-slave-0956"
server_key: "74dd00c537bab0434a7ae6c3791288fc"
server_key_iv: "ec2f15530b79996e484566cbb54339c5"
server_key_version: "encryptionkey@0"
I20260618 22:32:26.200650 873 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260618 22:32:26.204890 888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:26.206828 873 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260618 22:32:26.207037 873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "3e1bc07964014ec3ab8d453fb199e2fb"
format_stamp: "Formatted at 2026-06-18 22:32:26 on dist-test-slave-0956"
server_key: "74dd00c537bab0434a7ae6c3791288fc"
server_key_iv: "ec2f15530b79996e484566cbb54339c5"
server_key_version: "encryptionkey@0"
I20260618 22:32:26.207257 873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:26.243453 873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:26.248109 873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:26.248413 873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:26.266749 873 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:44571
I20260618 22:32:26.266777 939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:44571 every 8 connection(s)
I20260618 22:32:26.268178 873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:26.269461 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 873
I20260618 22:32:26.269726 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:32:26.270296 11211 external_mini_cluster.cc:1428] Setting key 5ef72aef1d909a696050cce95338a2d6
I20260618 22:32:26.272921 940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:26.284106 943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:26.272171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60049 (local address 127.10.242.254:44571)
0618 22:32:26.272962 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:26.272976 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:26.273027 (+ 51us) server_negotiation.cc:407] Connection header received
0618 22:32:26.274685 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:26.274709 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:26.275106 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:26.275475 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:26.276088 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:26.277553 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:26.279384 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:26.280172 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:26.280342 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:26.281743 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:26.281751 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:26.281766 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:26.282366 (+ 600us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:26.282444 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:26.282475 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:26.282576 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:26.282797 (+ 221us) server_negotiation.cc:299] Negotiation successful
0618 22:32:26.283071 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":338,"threads_started":1}
I20260618 22:32:26.285213 940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb: Bootstrap starting.
I20260618 22:32:26.288168 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:32:26.288797 940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:26.290032 940 log.cc:824] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:26.293483 940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb: No bootstrap required, opened a new log
I20260618 22:32:26.299966 943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:26.293175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53405 (local address 127.10.242.254:44571)
0618 22:32:26.293347 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:26.293353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:26.293742 (+ 389us) server_negotiation.cc:407] Connection header received
0618 22:32:26.293897 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:26.293903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:26.293990 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:26.294104 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:26.294591 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:26.295666 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:26.296991 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:26.297698 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:26.297835 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:26.299136 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:26.299142 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:26.299147 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:26.299168 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:26.299215 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:26.299235 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:26.299335 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:26.299513 (+ 178us) server_negotiation.cc:299] Negotiation successful
0618 22:32:26.299609 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:32:26.303092 940 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1bc07964014ec3ab8d453fb199e2fb" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44571 } }
I20260618 22:32:26.303485 940 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:26.303577 940 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e1bc07964014ec3ab8d453fb199e2fb, State: Initialized, Role: FOLLOWER
I20260618 22:32:26.304234 940 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1bc07964014ec3ab8d453fb199e2fb" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44571 } }
I20260618 22:32:26.304503 940 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:26.304652 940 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:26.304823 940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:26.307025 940 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1bc07964014ec3ab8d453fb199e2fb" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44571 } }
I20260618 22:32:26.307493 940 leader_election.cc:302] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e1bc07964014ec3ab8d453fb199e2fb; no voters:
I20260618 22:32:26.308003 940 leader_election.cc:288] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:26.308147 950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:26.308781 950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb [term 1 LEADER]: Becoming Leader. State: Replica: 3e1bc07964014ec3ab8d453fb199e2fb, State: Running, Role: LEADER
I20260618 22:32:26.309379 950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1bc07964014ec3ab8d453fb199e2fb" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44571 } }
I20260618 22:32:26.310175 940 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:26.312223 952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e1bc07964014ec3ab8d453fb199e2fb. Latest consensus state: current_term: 1 leader_uuid: "3e1bc07964014ec3ab8d453fb199e2fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1bc07964014ec3ab8d453fb199e2fb" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44571 } } }
I20260618 22:32:26.312371 951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e1bc07964014ec3ab8d453fb199e2fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e1bc07964014ec3ab8d453fb199e2fb" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44571 } } }
I20260618 22:32:26.312635 952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:26.312662 951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:26.314452 958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:26.319032 958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:26.328233 958 catalog_manager.cc:1347] Generated new cluster ID: 059f9a8302524733b5ba08758aa7a946
I20260618 22:32:26.328357 958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:26.341815 958 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:26.343171 958 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:26.351756 958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e1bc07964014ec3ab8d453fb199e2fb: Generated new TSK 0
I20260618 22:32:26.352949 958 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:32:26.366148 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 873
W20260618 22:32:26.367545 935 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60049
W20260618 22:32:26.368362 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 873: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:32:26.368527 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 873
2026-06-18T22:32:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1824 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:26 dist-test-slave-0956 krb5kdc[971](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:26 dist-test-slave-0956 krb5kdc[971](info): set up 2 sockets
Jun 18 22:32:26 dist-test-slave-0956 krb5kdc[971](info): commencing operation
krb5kdc: starting...
W20260618 22:32:27.397387 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.996s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:32:27 dist-test-slave-0956 krb5kdc[971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:32:27.648885 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:44119
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:38183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:44119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:27.983314 984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:27.983646 984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:27.983795 984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:27.991151 984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:27.991302 984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:27.991384 984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:27.991465 984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:27.991526 984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:28.001832 984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:38183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:44119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:44119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:28.003178 984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:28.004522 984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:28.016191 989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:28.016163 990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:28.016116 992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:28.017329 984 server_base.cc:1034] running on GCE node
I20260618 22:32:28.018113 984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:28.019609 984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:28.020836 984 hybrid_clock.cc:648] HybridClock initialized: now 1781821948020789 us; error 45 us; skew 500 ppm
I20260618 22:32:28.021265 984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:28 dist-test-slave-0956 krb5kdc[971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821948, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:28.024618 984 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:32:28.026268 984 webserver.cc:460] Webserver started at http://127.10.242.254:45499/ using document root <none> and password file <none>
I20260618 22:32:28.026960 984 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:28.027083 984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:28.027390 984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:28.030467 984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "be9b5b342ccd451db613d71bb277df67"
format_stamp: "Formatted at 2026-06-18 22:32:28 on dist-test-slave-0956"
server_key: "1633acc78973b284b7f48ada1b6f9529"
server_key_iv: "73df5ec692f44c1c06347367293bc95c"
server_key_version: "encryptionkey@0"
I20260618 22:32:28.031246 984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "be9b5b342ccd451db613d71bb277df67"
format_stamp: "Formatted at 2026-06-18 22:32:28 on dist-test-slave-0956"
server_key: "1633acc78973b284b7f48ada1b6f9529"
server_key_iv: "73df5ec692f44c1c06347367293bc95c"
server_key_version: "encryptionkey@0"
I20260618 22:32:28.036695 984 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260618 22:32:28.040331 999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:28.041775 984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260618 22:32:28.041949 984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "be9b5b342ccd451db613d71bb277df67"
format_stamp: "Formatted at 2026-06-18 22:32:28 on dist-test-slave-0956"
server_key: "1633acc78973b284b7f48ada1b6f9529"
server_key_iv: "73df5ec692f44c1c06347367293bc95c"
server_key_version: "encryptionkey@0"
I20260618 22:32:28.042148 984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:28.047899 984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:28.052186 984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:28.052469 984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:28.069756 984 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:44119
I20260618 22:32:28.069808 1050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:44119 every 8 connection(s)
I20260618 22:32:28.071126 984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:28.073444 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 984
I20260618 22:32:28.073622 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:32:28.074129 11211 external_mini_cluster.cc:1428] Setting key 3c1986eda35998ae9ddea0f03145bf03
I20260618 22:32:28.075486 1051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:28.082554 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:28.075928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47025 (local address 127.10.242.254:44119)
0618 22:32:28.076859 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:28.076874 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:28.076923 (+ 49us) server_negotiation.cc:407] Connection header received
0618 22:32:28.079084 (+ 2161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:28.079112 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:28.079493 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:28.079845 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:28.080875 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:28.080882 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:28.080895 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:28.080910 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:28.081001 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:28.081023 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:28.081298 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:28.081399 (+ 101us) server_negotiation.cc:299] Negotiation successful
0618 22:32:28.081569 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":427,"threads_started":1}
I20260618 22:32:28.086068 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:32:28.088667 1051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67: Bootstrap starting.
I20260618 22:32:28.092398 1051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:28.092623 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:28.090225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40365 (local address 127.10.242.254:44119)
0618 22:32:28.090416 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:28.090421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:28.090713 (+ 292us) server_negotiation.cc:407] Connection header received
0618 22:32:28.090914 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:28.090918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:28.090967 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:28.091048 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:28.091922 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:28.091926 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:28.091929 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:32:28.091943 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:28.091979 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:32:28.091994 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:28.092075 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:28.092193 (+ 118us) server_negotiation.cc:299] Negotiation successful
0618 22:32:28.092278 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260618 22:32:28.093609 1051 log.cc:824] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:28.096500 1051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67: No bootstrap required, opened a new log
I20260618 22:32:28.104103 1051 raft_consensus.cc:348] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be9b5b342ccd451db613d71bb277df67" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44119 } }
I20260618 22:32:28.104552 1051 raft_consensus.cc:374] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:28.104660 1051 raft_consensus.cc:729] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9b5b342ccd451db613d71bb277df67, State: Initialized, Role: FOLLOWER
I20260618 22:32:28.105243 1051 consensus_queue.cc:260] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be9b5b342ccd451db613d71bb277df67" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44119 } }
I20260618 22:32:28.105455 1051 raft_consensus.cc:388] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:28.105554 1051 raft_consensus.cc:482] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:28.105680 1051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:28.107874 1051 raft_consensus.cc:504] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be9b5b342ccd451db613d71bb277df67" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44119 } }
I20260618 22:32:28.108484 1051 leader_election.cc:302] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be9b5b342ccd451db613d71bb277df67; no voters:
I20260618 22:32:28.109182 1051 leader_election.cc:288] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:28.109265 1061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:28.109882 1061 raft_consensus.cc:686] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67 [term 1 LEADER]: Becoming Leader. State: Replica: be9b5b342ccd451db613d71bb277df67, State: Running, Role: LEADER
I20260618 22:32:28.110446 1061 consensus_queue.cc:237] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be9b5b342ccd451db613d71bb277df67" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44119 } }
I20260618 22:32:28.111233 1051 sys_catalog.cc:564] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:28.113063 1063 sys_catalog.cc:455] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be9b5b342ccd451db613d71bb277df67. Latest consensus state: current_term: 1 leader_uuid: "be9b5b342ccd451db613d71bb277df67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be9b5b342ccd451db613d71bb277df67" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44119 } } }
I20260618 22:32:28.113545 1063 sys_catalog.cc:458] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:28.113063 1062 sys_catalog.cc:455] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be9b5b342ccd451db613d71bb277df67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be9b5b342ccd451db613d71bb277df67" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 44119 } } }
I20260618 22:32:28.114086 1062 sys_catalog.cc:458] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:28.116396 1066 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:28.123036 1066 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:28.132493 1066 catalog_manager.cc:1347] Generated new cluster ID: e143d731cbef4c6d88ece305f9daa996
I20260618 22:32:28.132628 1066 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:28.148809 1066 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:28.149286 1066 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:28.156322 1066 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be9b5b342ccd451db613d71bb277df67: Generated new TSK 0
I20260618 22:32:28.157171 1066 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:32:28.170360 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 984
W20260618 22:32:28.172170 1046 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47025
W20260618 22:32:28.172897 11211 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 984: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:32:28.173038 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 984
2026-06-18T22:32:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1804 ms)
[----------] 18 tests from AuthTokenIssuingTest (33088 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:28 dist-test-slave-0956 krb5kdc[1082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:28 dist-test-slave-0956 krb5kdc[1082](info): set up 2 sockets
Jun 18 22:32:28 dist-test-slave-0956 krb5kdc[1082](info): commencing operation
krb5kdc: starting...
W20260618 22:32:29.197492 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.008s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:32:29 dist-test-slave-0956 krb5kdc[1082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:32:29.446751 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:43309
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:43309,127.10.242.253:43623,127.10.242.252:45315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:29.769861 1095 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260618 22:32:29.770218 1095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:29.770339 1095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:29.770457 1095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:29.777959 1095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:29.778060 1095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:29.778122 1095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:29.778192 1095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:29.778275 1095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:29.788349 1095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:43309,127.10.242.253:43623,127.10.242.252:45315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:43309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:29.789724 1095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:29.791011 1095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:29.803184 1100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:29.803160 1103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:29.803218 1101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:29.804008 1095 server_base.cc:1034] running on GCE node
I20260618 22:32:29.804847 1095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:29.806386 1095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:29.807614 1095 hybrid_clock.cc:648] HybridClock initialized: now 1781821949807548 us; error 44 us; skew 500 ppm
I20260618 22:32:29.807941 1095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:29 dist-test-slave-0956 krb5kdc[1082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821949, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:29.815558 1095 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:32:29.817225 1095 webserver.cc:460] Webserver started at http://127.10.242.254:42425/ using document root <none> and password file <none>
I20260618 22:32:29.817974 1095 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:29.818081 1095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:29.818400 1095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:29.821516 1095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "de23079ab5f143b4bf4c3e93edcee563"
format_stamp: "Formatted at 2026-06-18 22:32:29 on dist-test-slave-0956"
server_key: "b05cdf1be0a9ae3673d53155801952aa"
server_key_iv: "5dc02e4a494b56497b3c37046e61d15a"
server_key_version: "encryptionkey@0"
I20260618 22:32:29.822325 1095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "de23079ab5f143b4bf4c3e93edcee563"
format_stamp: "Formatted at 2026-06-18 22:32:29 on dist-test-slave-0956"
server_key: "b05cdf1be0a9ae3673d53155801952aa"
server_key_iv: "5dc02e4a494b56497b3c37046e61d15a"
server_key_version: "encryptionkey@0"
I20260618 22:32:29.827534 1095 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:32:29.831045 1110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:29.832437 1095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260618 22:32:29.832623 1095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "de23079ab5f143b4bf4c3e93edcee563"
format_stamp: "Formatted at 2026-06-18 22:32:29 on dist-test-slave-0956"
server_key: "b05cdf1be0a9ae3673d53155801952aa"
server_key_iv: "5dc02e4a494b56497b3c37046e61d15a"
server_key_version: "encryptionkey@0"
I20260618 22:32:29.832840 1095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:29.876724 1095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:29.881423 1095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:29.881695 1095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:29.899127 1095 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:43309
I20260618 22:32:29.899144 1161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:43309 every 8 connection(s)
I20260618 22:32:29.900609 1095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:29.901216 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 1095
I20260618 22:32:29.901422 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:32:29.902220 11211 external_mini_cluster.cc:1428] Setting key 9a76f531ca83841c59ff1b7faa337880
I20260618 22:32:29.904597 1162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:29.912003 1162 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.242.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.253@KRBTEST.COM" created.
Jun 18 22:32:29 dist-test-slave-0956 krb5kdc[1082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821949, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:32:29.936637 1166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:29.915300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42536 (local address 127.10.242.254:43309)
0618 22:32:29.917224 (+ 1924us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:29.917229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:29.917272 (+ 43us) server_negotiation.cc:407] Connection header received
0618 22:32:29.919230 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:29.919237 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:29.919483 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:29.919641 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:29.921290 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:29.922500 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:29.924490 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:29.925114 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:29.925252 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:29.928903 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:29.928926 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:29.928944 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:29.928983 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:29.932275 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:29.933037 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:29.933046 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:29.933050 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:29.933291 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:29.933705 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:29.933710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:29.933713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:29.934384 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:29.934567 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:29.935169 (+ 602us) server_negotiation.cc:299] Negotiation successful
0618 22:32:29.935600 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":136,"server-negotiator.queue_time_us":1715,"spinlock_wait_cycles":25856,"thread_start_us":1464,"threads_started":1}
I20260618 22:32:29.936697 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:29.915300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:43309 (local address 127.0.0.1:42536)
0618 22:32:29.916206 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:29.916255 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:29.918774 (+ 2519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:29.919714 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:29.919725 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:29.920513 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:29.921008 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:29.921037 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:29.922661 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:29.922666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:29.924369 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:29.924383 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:29.925368 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:29.925374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:29.925494 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:29.926094 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:29.926116 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:29.928640 (+ 2524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:29.932437 (+ 3797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:29.932448 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:29.932458 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:29.932898 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:29.933416 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:29.933420 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:29.933422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:29.933606 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:29.934528 (+ 922us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:29.934535 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:29.934771 (+ 236us) client_negotiation.cc:769] Sending connection context
0618 22:32:29.934985 (+ 214us) client_negotiation.cc:240] Negotiation successful
0618 22:32:29.935422 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":369,"threads_started":1}
I20260618 22:32:29.939849 1162 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:32:29.940901 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:29.940444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:43623 (local address 127.0.0.1:60488)
0618 22:32:29.940627 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:29.940766 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260618 22:32:29.941174 1112 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.242.253:43623: connect: Connection refused (error 111)
I20260618 22:32:29.942726 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:29.942301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:43623 (local address 127.0.0.1:60502)
0618 22:32:29.942452 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:29.942539 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260618 22:32:29.942919 1162 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.253:43623: Network error: Client connection negotiation failed: client connection to 127.10.242.253:43623: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.253.keytab.
Entry for principal kudu/127.10.242.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.253.keytab.
I20260618 22:32:29.958801 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.253:43623
--webserver_interface=127.10.242.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.253.keytab
--principal=kudu/127.10.242.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:43309,127.10.242.253:43623,127.10.242.252:45315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:32:29.992311 1162 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } attempt: 1
I20260618 22:32:29.993628 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:29.993045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:43623 (local address 127.0.0.1:60508)
0618 22:32:29.993298 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:29.993433 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260618 22:32:29.994686 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:29.994285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:43623 (local address 127.0.0.1:60512)
0618 22:32:29.994443 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:29.994529 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260618 22:32:29.994869 1162 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.253:43623: Network error: Client connection negotiation failed: client connection to 127.10.242.253:43623: connect: Connection refused (error 111)
I20260618 22:32:30.063163 1162 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } attempt: 2
I20260618 22:32:30.064448 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.063879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:43623 (local address 127.0.0.1:60514)
0618 22:32:30.064147 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.064265 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260618 22:32:30.065610 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.065182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:43623 (local address 127.0.0.1:60528)
0618 22:32:30.065335 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.065439 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260618 22:32:30.065851 1162 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.253:43623: Network error: Client connection negotiation failed: client connection to 127.10.242.253:43623: connect: Connection refused (error 111)
I20260618 22:32:30.157199 1162 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } attempt: 3
I20260618 22:32:30.158571 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.157994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:43623 (local address 127.0.0.1:60540)
0618 22:32:30.158271 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.158394 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260618 22:32:30.159704 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.159278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:43623 (local address 127.0.0.1:60544)
0618 22:32:30.159439 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.159525 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260618 22:32:30.159924 1162 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.253:43623: Network error: Client connection negotiation failed: client connection to 127.10.242.253:43623: connect: Connection refused (error 111)
W20260618 22:32:30.295835 1169 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260618 22:32:30.296182 1169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:30.296303 1169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:30.296418 1169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260618 22:32:30.303171 1162 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } attempt: 4
W20260618 22:32:30.303916 1169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:30.304014 1169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:30.304141 1169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:30.304234 1169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:30.304322 1169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:30.304435 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.303822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:43623 (local address 127.0.0.1:60554)
0618 22:32:30.304153 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.304266 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172}
I20260618 22:32:30.305454 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.305052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:43623 (local address 127.0.0.1:60564)
0618 22:32:30.305216 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.305313 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260618 22:32:30.305624 1162 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.253:43623: Network error: Client connection negotiation failed: client connection to 127.10.242.253:43623: connect: Connection refused (error 111)
I20260618 22:32:30.314790 1169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:43309,127.10.242.253:43623,127.10.242.252:45315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.253.keytab
--principal=kudu/127.10.242.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.253:43623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.242.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:30.316075 1169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:30.317458 1169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:30.328660 1174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:30.328579 1177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:30.328637 1175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:30.329483 1169 server_base.cc:1034] running on GCE node
I20260618 22:32:30.330298 1169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:30.331678 1169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:30.332904 1169 hybrid_clock.cc:648] HybridClock initialized: now 1781821950332841 us; error 35 us; skew 500 ppm
I20260618 22:32:30.333261 1169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:30 dist-test-slave-0956 krb5kdc[1082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821950, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:30.336287 1169 init.cc:377] Logged in from keytab as kudu/127.10.242.253@KRBTEST.COM (short username kudu)
I20260618 22:32:30.337771 1169 webserver.cc:460] Webserver started at http://127.10.242.253:37715/ using document root <none> and password file <none>
I20260618 22:32:30.338472 1169 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:30.338570 1169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:30.338865 1169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:30.341877 1169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-1/data/instance:
uuid: "fa19ab7a8a654ec5a7d1027e65449a9c"
format_stamp: "Formatted at 2026-06-18 22:32:30 on dist-test-slave-0956"
server_key: "bae20974b0f56c671d621b75821b473d"
server_key_iv: "a3299105f2840363e231ce1342926a0d"
server_key_version: "encryptionkey@0"
I20260618 22:32:30.342662 1169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-1/wal/instance:
uuid: "fa19ab7a8a654ec5a7d1027e65449a9c"
format_stamp: "Formatted at 2026-06-18 22:32:30 on dist-test-slave-0956"
server_key: "bae20974b0f56c671d621b75821b473d"
server_key_iv: "a3299105f2840363e231ce1342926a0d"
server_key_version: "encryptionkey@0"
I20260618 22:32:30.347765 1169 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:32:30.351167 1184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:30.352591 1169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:32:30.352746 1169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-1/wal
uuid: "fa19ab7a8a654ec5a7d1027e65449a9c"
format_stamp: "Formatted at 2026-06-18 22:32:30 on dist-test-slave-0956"
server_key: "bae20974b0f56c671d621b75821b473d"
server_key_iv: "a3299105f2840363e231ce1342926a0d"
server_key_version: "encryptionkey@0"
I20260618 22:32:30.352941 1169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:30.379416 1169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:30.384403 1169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:30.384665 1169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:30.402144 1169 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.253:43623
I20260618 22:32:30.402163 1235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.253:43623 every 8 connection(s)
I20260618 22:32:30.403493 1169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-1/data/info.pb
I20260618 22:32:30.407195 1236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:30.413239 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 1169
I20260618 22:32:30.413439 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-1/wal/instance
I20260618 22:32:30.413972 11211 external_mini_cluster.cc:1428] Setting key 90c8235e9adf464d3748315fa8316d17
I20260618 22:32:30.413833 1236 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:32:30 dist-test-slave-0956 krb5kdc[1082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821950, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.253@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:32:30.436352 1166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.416998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42540 (local address 127.10.242.254:43309)
0618 22:32:30.417237 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:30.417243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:30.418618 (+ 1375us) server_negotiation.cc:407] Connection header received
0618 22:32:30.421018 (+ 2400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:30.421023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:30.421142 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:30.421269 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:30.423218 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.424147 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.425956 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.426380 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.426499 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:30.429929 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:30.429934 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:30.429937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:30.429972 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:30.432895 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:30.433878 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:30.433882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:30.433884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:30.434017 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:30.434458 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:30.434462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:30.434464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:30.434775 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:30.434938 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:30.435697 (+ 759us) server_negotiation.cc:299] Negotiation successful
0618 22:32:30.435955 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
WARNING: no policy specified for kudu/127.10.242.252@KRBTEST.COM; defaulting to no policy
I20260618 22:32:30.437431 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.417631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:43309 (local address 127.0.0.1:42540)
0618 22:32:30.418430 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.418482 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:30.420685 (+ 2203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:30.421516 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:30.421527 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:30.422225 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:30.423005 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:30.423032 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.424454 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.424460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:30.425790 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:30.425804 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.426609 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.426618 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:30.426856 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:30.427515 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:30.427538 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:30.429698 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:30.433061 (+ 3363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:30.433071 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:30.433127 (+ 56us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:30.433735 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:30.434131 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:30.434137 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:30.434141 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:30.434330 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:30.434910 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:30.434919 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:30.435298 (+ 379us) client_negotiation.cc:769] Sending connection context
0618 22:32:30.435582 (+ 284us) client_negotiation.cc:240] Negotiation successful
0618 22:32:30.436171 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":330,"threads_started":1}
I20260618 22:32:30.439183 1236 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.252@KRBTEST.COM" created.
Jun 18 22:32:30 dist-test-slave-0956 krb5kdc[1082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821950, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.253@KRBTEST.COM for kudu/127.10.242.253@KRBTEST.COM
I20260618 22:32:30.452808 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.440035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:43623 (local address 127.0.0.1:60568)
0618 22:32:30.440209 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.440225 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:30.440320 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:30.441173 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:30.441177 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:30.441614 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:30.441750 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:30.441758 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.443092 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.443097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:30.443924 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:30.443935 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.444723 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.444730 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:30.444912 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:30.445452 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:30.445469 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:30.447467 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:30.450469 (+ 3002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:30.450475 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:30.450480 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:30.450815 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:30.451138 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:30.451143 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:30.451147 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:30.451292 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:30.451861 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:30.451865 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:30.452004 (+ 139us) client_negotiation.cc:769] Sending connection context
0618 22:32:30.452120 (+ 116us) client_negotiation.cc:240] Negotiation successful
0618 22:32:30.452414 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260618 22:32:30.452895 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.440124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60568 (local address 127.10.242.253:43623)
0618 22:32:30.440745 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:30.440751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:30.440776 (+ 25us) server_negotiation.cc:407] Connection header received
0618 22:32:30.440822 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:30.440828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:30.441009 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:30.441167 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:30.441902 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.442966 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.444044 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.444572 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.444677 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:30.447656 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:30.447667 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:30.447680 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:30.447708 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:30.450341 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:30.450928 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:30.450934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:30.450936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:30.451043 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:30.451381 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:30.451387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:30.451390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:30.451748 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:30.451941 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:30.452267 (+ 326us) server_negotiation.cc:299] Negotiation successful
0618 22:32:30.452530 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":479,"thread_start_us":359,"threads_started":1}
I20260618 22:32:30.455202 1236 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:32:30.456522 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.455928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60514)
0618 22:32:30.456145 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.456314 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260618 22:32:30.456796 1186 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
I20260618 22:32:30.458544 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.458113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60520)
0618 22:32:30.458280 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.458399 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260618 22:32:30.458776 1236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:45315: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.252.keytab.
Entry for principal kudu/127.10.242.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.252.keytab.
I20260618 22:32:30.469710 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.252:45315
--webserver_interface=127.10.242.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.252.keytab
--principal=kudu/127.10.242.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:43309,127.10.242.253:43623,127.10.242.252:45315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:32:30.508117 1236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } attempt: 1
I20260618 22:32:30.509425 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.508860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60530)
0618 22:32:30.509071 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.509241 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260618 22:32:30.510458 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.510056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60538)
0618 22:32:30.510226 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.510318 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260618 22:32:30.510653 1236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:45315: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
I20260618 22:32:30.578874 1236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } attempt: 2
I20260618 22:32:30.580075 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.579534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60554)
0618 22:32:30.579771 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.579885 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260618 22:32:30.581223 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.580786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60570)
0618 22:32:30.580946 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.581038 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260618 22:32:30.581453 1236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:45315: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
I20260618 22:32:30.604871 1162 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } attempt: 5
Jun 18 22:32:30 dist-test-slave-0956 krb5kdc[1082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821949, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for kudu/127.10.242.253@KRBTEST.COM
I20260618 22:32:30.619095 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.605992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60576 (local address 127.10.242.253:43623)
0618 22:32:30.606183 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:30.606189 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:30.606211 (+ 22us) server_negotiation.cc:407] Connection header received
0618 22:32:30.606276 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:30.606281 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:30.606377 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:30.606496 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:30.607331 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.608430 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.609724 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.610402 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.610533 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:30.613780 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:30.613786 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:30.613789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:30.613828 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:30.616332 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:30.617203 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:30.617209 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:30.617212 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:30.617340 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:30.617754 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:30.617759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:30.617762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:30.618092 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:30.618236 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:30.618681 (+ 445us) server_negotiation.cc:299] Negotiation successful
0618 22:32:30.618882 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:32:30.619180 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.605680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:43623 (local address 127.0.0.1:60576)
0618 22:32:30.605869 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.605889 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:30.606027 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:30.606607 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:30.606611 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:30.606989 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:30.607158 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:30.607167 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.608600 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.608604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:30.609558 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:30.609572 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.610572 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.610577 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:30.610742 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:30.611360 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:30.611378 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:30.613561 (+ 2183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:30.616531 (+ 2970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:30.616537 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:30.616541 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:30.617068 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:30.617443 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:30.617447 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:30.617450 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:30.617619 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:30.618210 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:30.618215 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:30.618379 (+ 164us) client_negotiation.cc:769] Sending connection context
0618 22:32:30.618524 (+ 145us) client_negotiation.cc:240] Negotiation successful
0618 22:32:30.618845 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260618 22:32:30.620254 1162 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:32:30.621261 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.620858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60582)
0618 22:32:30.621017 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.621121 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260618 22:32:30.622113 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.621794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60594)
0618 22:32:30.621941 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.622008 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260618 22:32:30.622298 1162 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:45315: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
I20260618 22:32:30.672736 1236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } attempt: 3
I20260618 22:32:30.673519 1162 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } attempt: 1
I20260618 22:32:30.674150 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.673556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60610)
0618 22:32:30.673814 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.673939 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260618 22:32:30.674656 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.674107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60622)
0618 22:32:30.674346 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.674447 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260618 22:32:30.675567 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.675023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60628)
0618 22:32:30.675307 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.675409 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
W20260618 22:32:30.675829 1236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:45315: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
I20260618 22:32:30.675981 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.675351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60644)
0618 22:32:30.675545 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.675642 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260618 22:32:30.676249 1162 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:45315: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
I20260618 22:32:30.744493 1162 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } attempt: 2
I20260618 22:32:30.745707 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.745206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60650)
0618 22:32:30.745433 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.745522 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260618 22:32:30.746665 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.746352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60666)
0618 22:32:30.746496 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.746565 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260618 22:32:30.746865 1162 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:45315: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
W20260618 22:32:30.791538 1243 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260618 22:32:30.791903 1243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:30.792006 1243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:30.792117 1243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:30.799659 1243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:30.799782 1243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:30.799844 1243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:30.799914 1243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:30.799991 1243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:30.810194 1243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:43309,127.10.242.253:43623,127.10.242.252:45315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.252.keytab
--principal=kudu/127.10.242.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.252:45315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.242.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:30.811478 1243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:30.812844 1243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:32:30.819021 1236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } attempt: 4
I20260618 22:32:30.820109 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.819664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60678)
0618 22:32:30.819882 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.819963 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260618 22:32:30.821043 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.820739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60688)
0618 22:32:30.820873 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.820945 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260618 22:32:30.821297 1236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:45315: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
W20260618 22:32:30.824527 1248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:30.824440 1251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:30.824465 1249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:30.825238 1243 server_base.cc:1034] running on GCE node
I20260618 22:32:30.826037 1243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:30.827498 1243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:30.828745 1243 hybrid_clock.cc:648] HybridClock initialized: now 1781821950828664 us; error 56 us; skew 500 ppm
I20260618 22:32:30.829120 1243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:30 dist-test-slave-0956 krb5kdc[1082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821950, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:30.832173 1243 init.cc:377] Logged in from keytab as kudu/127.10.242.252@KRBTEST.COM (short username kudu)
I20260618 22:32:30.833712 1243 webserver.cc:460] Webserver started at http://127.10.242.252:40403/ using document root <none> and password file <none>
I20260618 22:32:30.834456 1243 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:30.834548 1243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:30.834862 1243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:30.838066 1243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-2/data/instance:
uuid: "a549228664424c8f88ffba61f1acc575"
format_stamp: "Formatted at 2026-06-18 22:32:30 on dist-test-slave-0956"
server_key: "aa6b9cb55f82076c046c1419bd16a7e4"
server_key_iv: "3b149fb3c802e0a4680da88cb0c9004b"
server_key_version: "encryptionkey@0"
I20260618 22:32:30.838865 1243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-2/wal/instance:
uuid: "a549228664424c8f88ffba61f1acc575"
format_stamp: "Formatted at 2026-06-18 22:32:30 on dist-test-slave-0956"
server_key: "aa6b9cb55f82076c046c1419bd16a7e4"
server_key_iv: "3b149fb3c802e0a4680da88cb0c9004b"
server_key_version: "encryptionkey@0"
I20260618 22:32:30.843997 1243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260618 22:32:30.848320 1258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:30.850662 1243 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260618 22:32:30.850863 1243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-2/wal
uuid: "a549228664424c8f88ffba61f1acc575"
format_stamp: "Formatted at 2026-06-18 22:32:30 on dist-test-slave-0956"
server_key: "aa6b9cb55f82076c046c1419bd16a7e4"
server_key_iv: "3b149fb3c802e0a4680da88cb0c9004b"
server_key_version: "encryptionkey@0"
I20260618 22:32:30.851135 1243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:30.853053 1162 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } attempt: 3
I20260618 22:32:30.854276 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.853666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60698)
0618 22:32:30.854004 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.854105 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260618 22:32:30.855341 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.854961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60702)
0618 22:32:30.855136 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.855229 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260618 22:32:30.855547 1162 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:45315: Network error: Client connection negotiation failed: client connection to 127.10.242.252:45315: connect: Connection refused (error 111)
I20260618 22:32:30.867933 1243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:30.872354 1243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:30.872625 1243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:30.888845 1243 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.252:45315
I20260618 22:32:30.888870 1309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.252:45315 every 8 connection(s)
I20260618 22:32:30.890280 1243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-2/data/info.pb
I20260618 22:32:30.893111 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 1243
I20260618 22:32:30.893321 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821604371818-11211-0/minicluster-data/master-2/wal/instance
I20260618 22:32:30.893818 11211 external_mini_cluster.cc:1428] Setting key 8041b69f75a82d462e463e33973c8dce
I20260618 22:32:30.894163 1310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:30.903520 1310 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:32:30 dist-test-slave-0956 krb5kdc[1082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.252@KRBTEST.COM
Jun 18 22:32:30 dist-test-slave-0956 krb5kdc[1082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821950, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.252@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:32:30.918964 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.895679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60706 (local address 127.10.242.252:45315)
0618 22:32:30.896486 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:30.896498 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:30.896543 (+ 45us) server_negotiation.cc:407] Connection header received
0618 22:32:30.898919 (+ 2376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:30.898952 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:30.899529 (+ 577us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:30.900051 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:30.901969 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.903894 (+ 1925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.905749 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.906851 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.907040 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:30.911549 (+ 4509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:30.911557 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:30.911568 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:30.911609 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:30.914692 (+ 3083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:30.915363 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:30.915372 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:30.915381 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:30.915579 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:30.916058 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:30.916061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:30.916064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:30.916684 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:30.916971 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:30.917497 (+ 526us) server_negotiation.cc:299] Negotiation successful
0618 22:32:30.917942 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":369,"threads_started":1}
I20260618 22:32:30.919801 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.906382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:43309 (local address 127.0.0.1:42544)
0618 22:32:30.907186 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.907232 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:30.907443 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:30.907797 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:30.907804 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:30.908390 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:30.908621 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:30.908629 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.909640 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.909644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:30.910918 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:30.910930 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.911636 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.911643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:30.911825 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:30.912443 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:30.912462 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:30.914677 (+ 2215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:30.917437 (+ 2760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:30.917448 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:30.917455 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:30.917952 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:30.918286 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:30.918291 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:30.918295 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:30.918440 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:30.918927 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:30.918935 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:30.919091 (+ 156us) client_negotiation.cc:769] Sending connection context
0618 22:32:30.919278 (+ 187us) client_negotiation.cc:240] Negotiation successful
0618 22:32:30.919527 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":408,"threads_started":1}
I20260618 22:32:30.919973 1166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.906331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42544 (local address 127.10.242.254:43309)
0618 22:32:30.906528 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:30.906534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:30.907312 (+ 778us) server_negotiation.cc:407] Connection header received
0618 22:32:30.907578 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:30.907583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:30.907645 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:30.907723 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:30.908761 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.909503 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.911045 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.911465 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.911570 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:30.914871 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:30.914876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:30.914879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:30.914912 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:30.917320 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:30.918054 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:30.918059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:30.918063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:30.918188 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:30.918535 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:30.918540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:30.918544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:30.918784 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:30.918926 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:30.919408 (+ 482us) server_negotiation.cc:299] Negotiation successful
0618 22:32:30.919644 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260618 22:32:30.921414 1310 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:32:30 dist-test-slave-0956 krb5kdc[1082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821950, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.252@KRBTEST.COM for kudu/127.10.242.253@KRBTEST.COM
Jun 18 22:32:30 dist-test-slave-0956 krb5kdc[1082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:32:30.935318 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.922290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60590 (local address 127.10.242.253:43623)
0618 22:32:30.922461 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:30.922465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:30.922482 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:32:30.922536 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:30.922542 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:30.922603 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:30.922678 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:30.923448 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.924246 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.925449 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.925860 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.925974 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:30.929508 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:30.929513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:30.929516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:30.929549 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:30.932639 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:30.933365 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:30.933369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:30.933371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:30.933552 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:30.933931 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:30.933936 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:30.933939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:30.934281 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:30.934431 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:30.934862 (+ 431us) server_negotiation.cc:299] Negotiation successful
0618 22:32:30.935067 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260618 22:32:30.935341 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.922174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:43623 (local address 127.0.0.1:60590)
0618 22:32:30.922342 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.922358 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:30.922446 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:30.922707 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:30.922712 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:30.923144 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:30.923320 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:30.923327 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.924364 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.924369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:30.925314 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:30.925325 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.925993 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.925999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:30.926160 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:30.926778 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:30.926799 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:30.929254 (+ 2455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:30.932799 (+ 3545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:30.932803 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:30.932806 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:30.933246 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:30.933666 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:30.933669 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:30.933671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:30.933818 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:30.934420 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:30.934424 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:30.934580 (+ 156us) client_negotiation.cc:769] Sending connection context
0618 22:32:30.934729 (+ 149us) client_negotiation.cc:240] Negotiation successful
0618 22:32:30.934994 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260618 22:32:30.936357 1310 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:32:30.937809 1166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.922123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42552 (local address 127.10.242.254:43309)
0618 22:32:30.922883 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:30.922888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:30.922904 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:32:30.922947 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:30.922951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:30.923015 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:30.923106 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:30.924394 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.925240 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.926586 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.927138 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.927261 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:30.931846 (+ 4585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:30.931853 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:30.931857 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:30.931900 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:30.934779 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:30.935696 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:30.935700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:30.935703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:30.935853 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:30.936286 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:30.936290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:30.936293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:30.936642 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:30.936911 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:30.937197 (+ 286us) server_negotiation.cc:299] Negotiation successful
0618 22:32:30.937429 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653}
Jun 18 22:32:30 dist-test-slave-0956 krb5kdc[1082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821950, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.252@KRBTEST.COM for kudu/127.10.242.252@KRBTEST.COM
Jun 18 22:32:30 dist-test-slave-0956 krb5kdc[1082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.253@KRBTEST.COM
I20260618 22:32:30.950026 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.937195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60720)
0618 22:32:30.937524 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:30.937542 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:30.937671 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:30.938114 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:30.938117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:30.938485 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:30.938613 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:30.938622 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.939650 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.939655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:30.940850 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:30.940866 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.941623 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.941629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:30.941800 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:30.942350 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:30.942366 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:30.944929 (+ 2563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:30.947654 (+ 2725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:30.947660 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:30.947664 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:30.948013 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:30.948350 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:30.948355 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:30.948359 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:30.948477 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:30.948979 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:30.948983 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:30.949197 (+ 214us) client_negotiation.cc:769] Sending connection context
0618 22:32:30.949315 (+ 118us) client_negotiation.cc:240] Negotiation successful
0618 22:32:30.949628 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":1}
I20260618 22:32:30.950078 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.937323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60720 (local address 127.10.242.252:45315)
0618 22:32:30.937489 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:30.937495 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:30.937607 (+ 112us) server_negotiation.cc:407] Connection header received
0618 22:32:30.937790 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:30.937795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:30.937868 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:30.937975 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:30.938735 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.939540 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.940998 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.941459 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.941561 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:30.945140 (+ 3579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:30.945145 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:30.945148 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:30.945176 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:30.947495 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:30.948131 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:30.948136 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:30.948140 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:30.948255 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:30.948571 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:30.948576 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:30.948580 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:30.948876 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:30.949003 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:30.949402 (+ 399us) server_negotiation.cc:299] Negotiation successful
0618 22:32:30.949794 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":161,"server-negotiator.queue_time_us":62}
I20260618 22:32:30.954623 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:30.939861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60600 (local address 127.10.242.253:43623)
0618 22:32:30.940005 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:30.940009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:30.940024 (+ 15us) server_negotiation.cc:407] Connection header received
0618 22:32:30.940067 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:30.940071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:30.940123 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:30.940197 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:30.941632 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.942351 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.943977 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:30.944589 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:30.944684 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:30.949134 (+ 4450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:30.949144 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:30.949150 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:30.949191 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:30.951992 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:30.952689 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:30.952695 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:30.952698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:30.952836 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:30.953267 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:30.953272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:30.953275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:30.953544 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:30.953790 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:30.954088 (+ 298us) server_negotiation.cc:299] Negotiation successful
0618 22:32:30.954317 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260618 22:32:30.954945 1310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575: Bootstrap starting.
I20260618 22:32:30.958770 1310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:30.959679 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:32:30.960047 1310 log.cc:824] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:30.963212 1310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575: No bootstrap required, opened a new log
I20260618 22:32:30.962863 1137 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "de23079ab5f143b4bf4c3e93edcee563"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42552
I20260618 22:32:30.974073 1310 raft_consensus.cc:348] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } } peers { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } } peers { permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } }
I20260618 22:32:30.974524 1310 raft_consensus.cc:729] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a549228664424c8f88ffba61f1acc575, State: Initialized, Role: FOLLOWER
I20260618 22:32:30.975097 1310 consensus_queue.cc:260] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } } peers { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } } peers { permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } }
I20260618 22:32:30.976052 1316 sys_catalog.cc:455] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } } peers { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } } peers { permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } } }
I20260618 22:32:30.976361 1316 sys_catalog.cc:458] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:32:30.977396 1310 sys_catalog.cc:564] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575 [sys.catalog]: configured and running, proceeding with master startup.
W20260618 22:32:30.986807 1327 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:32:30.986963 1327 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:32:30.984726 1243 master_runner.cc:186] Error getting master registration for 127.10.242.254:43309: OK, instance_id { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" instance_seqno: 1781821949894233 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 22:32:30.995786 1243 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260618 22:32:31.032815 1162 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } attempt: 4
Jun 18 22:32:31 dist-test-slave-0956 krb5kdc[1082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821949, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for kudu/127.10.242.252@KRBTEST.COM
I20260618 22:32:31.048686 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:31.033734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60726 (local address 127.10.242.252:45315)
0618 22:32:31.033945 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.033949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.033969 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:32:31.034162 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.034166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.034228 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.034311 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:31.035271 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.036494 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.038504 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.039233 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.039366 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:31.043350 (+ 3984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:31.043362 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:31.043365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:31.043400 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:31.046109 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:31.046751 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:31.046755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:31.046757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:31.046878 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:31.047230 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:31.047235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:31.047238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:31.047588 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:31.047746 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:31.048126 (+ 380us) server_negotiation.cc:299] Negotiation successful
0618 22:32:31.048326 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260618 22:32:31.048720 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:31.033669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60726)
0618 22:32:31.033891 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:31.033913 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:31.034045 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:31.034368 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:31.034373 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:31.034869 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:31.035090 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:31.035102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.036640 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.036646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:31.038324 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:31.038343 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.039408 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.039414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:31.039651 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:31.040383 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:31.040408 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:31.043134 (+ 2726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:31.046248 (+ 3114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:31.046254 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:31.046258 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:31.046636 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:31.046981 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:31.046986 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:31.046991 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:31.047132 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:31.047707 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:31.047712 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:31.047881 (+ 169us) client_negotiation.cc:769] Sending connection context
0618 22:32:31.048009 (+ 128us) client_negotiation.cc:240] Negotiation successful
0618 22:32:31.048310 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260618 22:32:31.054860 1162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563: Bootstrap starting.
I20260618 22:32:31.059309 1162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:31.060690 1162 log.cc:824] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:31.064177 1162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563: No bootstrap required, opened a new log
I20260618 22:32:31.071951 1162 raft_consensus.cc:348] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } } peers { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } } peers { permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } }
I20260618 22:32:31.072412 1162 raft_consensus.cc:729] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de23079ab5f143b4bf4c3e93edcee563, State: Initialized, Role: FOLLOWER
I20260618 22:32:31.072942 1162 consensus_queue.cc:260] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } } peers { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } } peers { permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } }
I20260618 22:32:31.074115 1330 sys_catalog.cc:455] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } } peers { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } } peers { permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } } }
I20260618 22:32:31.074498 1330 sys_catalog.cc:458] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:32:31.075440 1162 sys_catalog.cc:564] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:31.075506 1137 raft_consensus.cc:482] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260618 22:32:31.075680 1137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:31.078264 1137 raft_consensus.cc:504] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } } peers { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } } peers { permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } }
I20260618 22:32:31.079507 1137 leader_election.cc:288] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [CANDIDATE]: Term 1 election: Requested vote from peers fa19ab7a8a654ec5a7d1027e65449a9c (127.10.242.253:43623), a549228664424c8f88ffba61f1acc575 (127.10.242.252:45315)
I20260618 22:32:31.080441 1285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de23079ab5f143b4bf4c3e93edcee563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a549228664424c8f88ffba61f1acc575"
I20260618 22:32:31.080940 1285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:31.084357 1285 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de23079ab5f143b4bf4c3e93edcee563 in term 1.
I20260618 22:32:31.085067 1112 leader_election.cc:302] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a549228664424c8f88ffba61f1acc575, de23079ab5f143b4bf4c3e93edcee563; no voters:
I20260618 22:32:31.085546 1330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:31.086090 1330 raft_consensus.cc:686] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [term 1 LEADER]: Becoming Leader. State: Replica: de23079ab5f143b4bf4c3e93edcee563, State: Running, Role: LEADER
I20260618 22:32:31.086634 1330 consensus_queue.cc:237] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } } peers { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } } peers { permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } }
I20260618 22:32:31.098007 1333 sys_catalog.cc:455] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de23079ab5f143b4bf4c3e93edcee563. Latest consensus state: current_term: 1 leader_uuid: "de23079ab5f143b4bf4c3e93edcee563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } } peers { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } } peers { permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } } }
I20260618 22:32:31.098425 1333 sys_catalog.cc:458] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:31.099197 1341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:31.117175 1341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:31.120509 1236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } attempt: 5
W20260618 22:32:31.124751 1264 tablet.cc:2369] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:32:31.134213 1285 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575 [term 1 FOLLOWER]: Refusing update from remote peer de23079ab5f143b4bf4c3e93edcee563: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260618 22:32:31.134946 1116 tablet.cc:2369] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:32:31.135319 1330 consensus_queue.cc:1035] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:31.146962 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:31.100050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60604 (local address 127.10.242.253:43623)
0618 22:32:31.100245 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.100252 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.119210 (+ 18958us) server_negotiation.cc:407] Connection header received
0618 22:32:31.120479 (+ 1269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.120490 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.120581 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.120701 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:31.130908 (+ 10207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.131877 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.134999 (+ 3122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.135574 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.135668 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:31.139152 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:31.139158 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:31.139163 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:31.139210 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:31.143504 (+ 4294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:31.144387 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:31.144394 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:31.144398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:31.144586 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:31.145050 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:31.145056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:31.145059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:31.145488 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:31.145733 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:31.146315 (+ 582us) server_negotiation.cc:299] Negotiation successful
0618 22:32:31.146591 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:32:31.152540 1166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:31.100370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42560 (local address 127.10.242.254:43309)
0618 22:32:31.100537 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.100543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.111476 (+ 10933us) server_negotiation.cc:407] Connection header received
0618 22:32:31.114974 (+ 3498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.114981 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.115080 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.115158 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:31.125659 (+ 10501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.126763 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.139548 (+ 12785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.140573 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.140713 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:31.143963 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:31.143970 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:31.143975 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:31.144045 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:31.148420 (+ 4375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:31.149307 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:31.149314 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:31.149318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:31.149486 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:31.149893 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:31.149898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:31.149901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:31.150280 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:31.150552 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:31.151917 (+ 1365us) server_negotiation.cc:299] Negotiation successful
0618 22:32:31.152155 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:32:31.161366 1316 sys_catalog.cc:455] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de23079ab5f143b4bf4c3e93edcee563. Latest consensus state: current_term: 1 leader_uuid: "de23079ab5f143b4bf4c3e93edcee563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } } peers { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } } peers { permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } } }
I20260618 22:32:31.161729 1316 sys_catalog.cc:458] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:32:31.164798 1348 mvcc.cc:205] Tried to move back new op lower bound from 7298342711830917120 to 7298342711655886848. Current Snapshot: MvccSnapshot[applied={T|T < 7298342711830917120}]
I20260618 22:32:31.169821 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:31.112096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60736 (local address 127.10.242.252:45315)
0618 22:32:31.112298 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.112303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.122238 (+ 9935us) server_negotiation.cc:407] Connection header received
0618 22:32:31.122505 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.122511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.122600 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.122684 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:31.133770 (+ 11086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.134808 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.139098 (+ 4290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.150089 (+ 10991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.150222 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:31.153417 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:31.153425 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:31.153430 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:31.153470 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:31.159751 (+ 6281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:31.166131 (+ 6380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:31.166139 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:31.166144 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:31.166361 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:31.166862 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:31.166867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:31.166871 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:31.167242 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:31.167517 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:31.169128 (+ 1611us) server_negotiation.cc:299] Negotiation successful
0618 22:32:31.169412 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
Jun 18 22:32:31 dist-test-slave-0956 krb5kdc[1082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821950, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.253@KRBTEST.COM for kudu/127.10.242.252@KRBTEST.COM
I20260618 22:32:31.177896 1316 sys_catalog.cc:455] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "de23079ab5f143b4bf4c3e93edcee563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } } peers { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } } peers { permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } } }
I20260618 22:32:31.178153 1316 sys_catalog.cc:458] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:32:31.188304 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:31.121132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:45315 (local address 127.0.0.1:60750)
0618 22:32:31.121397 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:31.121415 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:31.121519 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:31.126581 (+ 5062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:31.126588 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:31.127160 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:31.127382 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:31.127394 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.155719 (+ 28325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.155726 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:31.157268 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:31.157293 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.168124 (+ 10831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.168133 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:31.168409 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:31.169237 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:31.169266 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:31.172335 (+ 3069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:31.183442 (+ 11107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:31.183447 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:31.183451 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:31.184003 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:31.185358 (+ 1355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:31.185369 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:31.185373 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:31.185902 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:31.187173 (+ 1271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:31.187179 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:31.187375 (+ 196us) client_negotiation.cc:769] Sending connection context
0618 22:32:31.187522 (+ 147us) client_negotiation.cc:240] Negotiation successful
0618 22:32:31.187880 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260618 22:32:31.189440 1347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:31.121217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60750 (local address 127.10.242.252:45315)
0618 22:32:31.126045 (+ 4828us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.126053 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.126079 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:32:31.126187 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.126193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.126325 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.127598 (+ 1273us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:31.128057 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.155506 (+ 27449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.157512 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.158306 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.168796 (+ 10490us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:31.172674 (+ 3878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:31.172682 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:31.172687 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:31.172731 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:31.183220 (+ 10489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:31.184251 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:31.184258 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:31.184261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:31.184740 (+ 479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:31.186237 (+ 1497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:31.186245 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:31.186249 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:31.187027 (+ 778us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:31.188595 (+ 1568us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:31.188748 (+ 153us) server_negotiation.cc:299] Negotiation successful
0618 22:32:31.189004 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4582,"thread_start_us":3081,"threads_started":1}
I20260618 22:32:31.196420 1236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c: Bootstrap starting.
I20260618 22:32:31.200752 1236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:31.201423 1330 sys_catalog.cc:455] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "de23079ab5f143b4bf4c3e93edcee563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } } peers { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } } peers { permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } } }
I20260618 22:32:31.201730 1330 sys_catalog.cc:458] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:31.192382 1095 master_runner.cc:186] Error getting master registration for 127.10.242.253:43623: OK, instance_id { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" instance_seqno: 1781821950397015 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 22:32:31.203095 1236 log.cc:824] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:31.203156 1095 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260618 22:32:31.205420 1341 catalog_manager.cc:1347] Generated new cluster ID: f069a2338d784a52836343243a03f1ac
I20260618 22:32:31.205523 1341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:31.223903 1236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c: No bootstrap required, opened a new log
I20260618 22:32:31.232453 1341 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:31.234493 1341 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:31.234220 1236 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } } peers { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } } peers { permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } }
I20260618 22:32:31.234642 1236 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa19ab7a8a654ec5a7d1027e65449a9c, State: Initialized, Role: FOLLOWER
I20260618 22:32:31.235172 1236 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } } peers { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } } peers { permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } }
I20260618 22:32:31.242139 1360 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } } peers { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } } peers { permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } } }
I20260618 22:32:31.244302 1360 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:32:31.244158 1236 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:31.244889 1210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:31.250922 1211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de23079ab5f143b4bf4c3e93edcee563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c"
I20260618 22:32:31.251472 1341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563: Generated new TSK 0
I20260618 22:32:31.252175 1210 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c [term 1 FOLLOWER]: Refusing update from remote peer de23079ab5f143b4bf4c3e93edcee563: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:32:31.252739 1341 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:32:31.257280 1211 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de23079ab5f143b4bf4c3e93edcee563 in term 1.
I20260618 22:32:31.257266 1333 consensus_queue.cc:1035] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:31.290376 1360 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader de23079ab5f143b4bf4c3e93edcee563. Latest consensus state: current_term: 1 leader_uuid: "de23079ab5f143b4bf4c3e93edcee563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } } peers { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } } peers { permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } } }
I20260618 22:32:31.290706 1360 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:32:31.292577 1372 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "de23079ab5f143b4bf4c3e93edcee563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } } peers { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } } peers { permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } } }
I20260618 22:32:31.293033 1372 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:32:31.308007 1377 catalog_manager.cc:1259] Loaded cluster ID: f069a2338d784a52836343243a03f1ac
I20260618 22:32:31.308209 1377 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c: loading cluster ID for follower catalog manager: success
I20260618 22:32:31.311980 1377 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c: acquiring CA information for follower catalog manager: success
I20260618 22:32:31.319794 1377 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fa19ab7a8a654ec5a7d1027e65449a9c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260618 22:32:31.421696 1240 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:31.401934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60618 (local address 127.10.242.253:43623)
0618 22:32:31.402123 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.402130 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.414435 (+ 12305us) server_negotiation.cc:407] Connection header received
0618 22:32:31.416966 (+ 2531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.416974 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.417115 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.417195 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:31.421448 (+ 4253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60618: BlockingRecv error: recv got EOF from 127.0.0.1:60618 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260618 22:32:31.433051 1166 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:31.410607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42566 (local address 127.10.242.254:43309)
0618 22:32:31.410831 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.410837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.429991 (+ 19154us) server_negotiation.cc:407] Connection header received
0618 22:32:31.431530 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.431536 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.431622 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.431699 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:31.432791 (+ 1092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42566: BlockingRecv error: recv got EOF from 127.0.0.1:42566 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260618 22:32:31.446550 1347 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:31.415009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60756 (local address 127.10.242.252:45315)
0618 22:32:31.415194 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.415200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.430727 (+ 15527us) server_negotiation.cc:407] Connection header received
0618 22:32:31.430905 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.430911 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.431038 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.431156 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:31.446297 (+ 15141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60756: BlockingRecv error: recv got EOF from 127.0.0.1:60756 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:32:31.499135 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:31.484705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60632 (local address 127.10.242.253:43623)
0618 22:32:31.484948 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.484955 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.485486 (+ 531us) server_negotiation.cc:407] Connection header received
0618 22:32:31.485842 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.485850 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.486115 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.486261 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:31.486994 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.488195 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.490311 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.491005 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.491108 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:31.497518 (+ 6410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:31.498269 (+ 751us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:31.498397 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:31.498663 (+ 266us) server_negotiation.cc:299] Negotiation successful
0618 22:32:31.498774 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260618 22:32:31.558562 1347 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:31.546849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60762 (local address 127.10.242.252:45315)
0618 22:32:31.547047 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.547053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.556434 (+ 9381us) server_negotiation.cc:407] Connection header received
0618 22:32:31.556750 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.556756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.556898 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.557014 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:31.558281 (+ 1267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60762: BlockingRecv error: recv got EOF from 127.0.0.1:60762 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260618 22:32:31.611048 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:31.592829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60646 (local address 127.10.242.253:43623)
0618 22:32:31.593031 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.593038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.604294 (+ 11256us) server_negotiation.cc:407] Connection header received
0618 22:32:31.604639 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.604647 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.604738 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.604804 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:31.605491 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.606647 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.608687 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.609428 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.609545 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:31.610160 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:31.610344 (+ 184us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:31.610406 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:31.610633 (+ 227us) server_negotiation.cc:299] Negotiation successful
0618 22:32:31.610723 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260618 22:32:31.658390 1347 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:31.645123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60776 (local address 127.10.242.252:45315)
0618 22:32:31.645356 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.645362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.656406 (+ 11044us) server_negotiation.cc:407] Connection header received
0618 22:32:31.656701 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.656708 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.656811 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.656895 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:31.658152 (+ 1257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60776: BlockingRecv error: recv got EOF from 127.0.0.1:60776 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260618 22:32:31.681310 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:31.673950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60656 (local address 127.10.242.253:43623)
0618 22:32:31.674138 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.674143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.674691 (+ 548us) server_negotiation.cc:407] Connection header received
0618 22:32:31.674914 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.674920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.675023 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.675132 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:31.675659 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.676905 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.679006 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.679706 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.679854 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:31.680372 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:31.680542 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:31.680646 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:31.680836 (+ 190us) server_negotiation.cc:299] Negotiation successful
0618 22:32:31.680943 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260618 22:32:31.692497 1347 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:31.690200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60786 (local address 127.10.242.252:45315)
0618 22:32:31.690395 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.690401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.690945 (+ 544us) server_negotiation.cc:407] Connection header received
0618 22:32:31.691158 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.691164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.691270 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.691378 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:31.692247 (+ 869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60786: BlockingRecv error: recv got EOF from 127.0.0.1:60786 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260618 22:32:31.707558 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:31.700664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60672 (local address 127.10.242.253:43623)
0618 22:32:31.700930 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.700936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.701506 (+ 570us) server_negotiation.cc:407] Connection header received
0618 22:32:31.701706 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.701712 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.701796 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.701900 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:31.702394 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.703602 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.705537 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.706187 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.706333 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:31.706809 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:31.706941 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:31.707031 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:31.707178 (+ 147us) server_negotiation.cc:299] Negotiation successful
0618 22:32:31.707277 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
W20260618 22:32:31.723676 1347 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:31.714984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60800 (local address 127.10.242.252:45315)
0618 22:32:31.715205 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.715211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.721830 (+ 6619us) server_negotiation.cc:407] Connection header received
0618 22:32:31.722082 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.722089 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.722190 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.722266 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:31.723445 (+ 1179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60800: BlockingRecv error: recv got EOF from 127.0.0.1:60800 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
I20260618 22:32:31.745064 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:31.738182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60678 (local address 127.10.242.253:43623)
0618 22:32:31.738360 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.738366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.738893 (+ 527us) server_negotiation.cc:407] Connection header received
0618 22:32:31.739104 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.739111 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.739205 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.739334 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:31.739788 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.740893 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.742909 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.743574 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.743723 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:31.744202 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:31.744403 (+ 201us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:31.744509 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:31.744652 (+ 143us) server_negotiation.cc:299] Negotiation successful
0618 22:32:31.744744 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260618 22:32:31.759034 1347 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:31.756703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60804 (local address 127.10.242.252:45315)
0618 22:32:31.756892 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.756898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.757500 (+ 602us) server_negotiation.cc:407] Connection header received
0618 22:32:31.757711 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.757716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.757808 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.757912 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:31.758786 (+ 874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60804: BlockingRecv error: recv got EOF from 127.0.0.1:60804 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:32:31.774307 1333 sys_catalog.cc:455] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "de23079ab5f143b4bf4c3e93edcee563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de23079ab5f143b4bf4c3e93edcee563" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 43309 } } peers { permanent_uuid: "fa19ab7a8a654ec5a7d1027e65449a9c" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 43623 } } peers { permanent_uuid: "a549228664424c8f88ffba61f1acc575" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 45315 } } }
I20260618 22:32:31.774678 1333 sys_catalog.cc:458] T 00000000000000000000000000000000 P de23079ab5f143b4bf4c3e93edcee563 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:31.790870 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:31.784832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60692 (local address 127.10.242.253:43623)
0618 22:32:31.785038 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.785044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.785388 (+ 344us) server_negotiation.cc:407] Connection header received
0618 22:32:31.785555 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.785562 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.785653 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.785774 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:31.786175 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.787405 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.788883 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.789557 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.789706 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:31.790097 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:31.790266 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:31.790363 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:31.790490 (+ 127us) server_negotiation.cc:299] Negotiation successful
0618 22:32:31.790588 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260618 22:32:31.804455 1347 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:31.802282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60808 (local address 127.10.242.252:45315)
0618 22:32:31.802481 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.802486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.802827 (+ 341us) server_negotiation.cc:407] Connection header received
0618 22:32:31.803030 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.803034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.803156 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.803259 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:31.804220 (+ 961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60808: BlockingRecv error: recv got EOF from 127.0.0.1:60808 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260618 22:32:31.845840 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:31.840077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60700 (local address 127.10.242.253:43623)
0618 22:32:31.840289 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.840295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.840573 (+ 278us) server_negotiation.cc:407] Connection header received
0618 22:32:31.840735 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.840741 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.840862 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.840964 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:31.841405 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.842482 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.843805 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.844486 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.844617 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:31.845027 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:31.845208 (+ 181us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:31.845302 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:31.845454 (+ 152us) server_negotiation.cc:299] Negotiation successful
0618 22:32:31.845534 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260618 22:32:31.855265 1347 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:31.851775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60820 (local address 127.10.242.252:45315)
0618 22:32:31.851930 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.851934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.853739 (+ 1805us) server_negotiation.cc:407] Connection header received
0618 22:32:31.853947 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.853954 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.854045 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.854144 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:31.854983 (+ 839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60820: BlockingRecv error: recv got EOF from 127.0.0.1:60820 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260618 22:32:31.927541 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:31.922533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60706 (local address 127.10.242.253:43623)
0618 22:32:31.922704 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.922708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.922997 (+ 289us) server_negotiation.cc:407] Connection header received
0618 22:32:31.923164 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.923168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.923239 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.923325 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:31.923651 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.924369 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.925874 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:31.926454 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:31.926554 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:31.926903 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:31.927016 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:31.927081 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:31.927263 (+ 182us) server_negotiation.cc:299] Negotiation successful
0618 22:32:31.927328 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260618 22:32:31.934697 1347 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:31.932931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60832 (local address 127.10.242.252:45315)
0618 22:32:31.933164 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:31.933171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:31.933458 (+ 287us) server_negotiation.cc:407] Connection header received
0618 22:32:31.933641 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:31.933645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:31.933717 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:31.933808 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:31.934525 (+ 717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60832: BlockingRecv error: recv got EOF from 127.0.0.1:60832 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260618 22:32:31.989107 1327 catalog_manager.cc:1259] Loaded cluster ID: f069a2338d784a52836343243a03f1ac
I20260618 22:32:31.989249 1327 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575: loading cluster ID for follower catalog manager: success
I20260618 22:32:31.991864 1327 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575: acquiring CA information for follower catalog manager: success
I20260618 22:32:31.993855 1327 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a549228664424c8f88ffba61f1acc575: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 22:32:32.070847 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:32.066126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60716 (local address 127.10.242.253:43623)
0618 22:32:32.066298 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:32.066303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:32.066613 (+ 310us) server_negotiation.cc:407] Connection header received
0618 22:32:32.066718 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:32.066723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:32.066805 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:32.066891 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:32.067286 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:32.067992 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:32.069367 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:32.069791 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:32.069884 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:32.070272 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:32.070388 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:32.070455 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:32.070553 (+ 98us) server_negotiation.cc:299] Negotiation successful
0618 22:32:32.070616 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:32:32.082753 1347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:32.076585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60842 (local address 127.10.242.252:45315)
0618 22:32:32.076817 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:32.076824 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:32.077072 (+ 248us) server_negotiation.cc:407] Connection header received
0618 22:32:32.077386 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:32.077391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:32.077556 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:32.077666 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:32.078128 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:32.078881 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:32.080362 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:32.080890 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:32.081010 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:32.081559 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:32.082099 (+ 540us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:32.082246 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:32.082366 (+ 120us) server_negotiation.cc:299] Negotiation successful
0618 22:32:32.082455 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260618 22:32:32.095618 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:32.088604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60726 (local address 127.10.242.253:43623)
0618 22:32:32.088812 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:32.088819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:32.089799 (+ 980us) server_negotiation.cc:407] Connection header received
0618 22:32:32.090065 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:32.090070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:32.090161 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:32.090225 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:32.090954 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:32.091986 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:32.093858 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:32.094283 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:32.094394 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:32.094870 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:32.094998 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:32.095088 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:32.095245 (+ 157us) server_negotiation.cc:299] Negotiation successful
0618 22:32:32.095331 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:32:32.097004 1485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:32.088462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42582 (local address 127.10.242.254:43309)
0618 22:32:32.089252 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:32.089258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:32.089751 (+ 493us) server_negotiation.cc:407] Connection header received
0618 22:32:32.089968 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:32.089973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:32.090179 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:32.091092 (+ 913us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:32.091648 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:32.092957 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:32.094639 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:32.095129 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:32.095690 (+ 561us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:32.095906 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:32.096342 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:32.096449 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:32.096595 (+ 146us) server_negotiation.cc:299] Negotiation successful
0618 22:32:32.096684 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":506,"threads_started":1}
I20260618 22:32:32.098120 1347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:32.090198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60852 (local address 127.10.242.252:45315)
0618 22:32:32.090445 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:32.090450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:32.090496 (+ 46us) server_negotiation.cc:407] Connection header received
0618 22:32:32.090592 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:32.090597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:32.090762 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:32.090853 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:32.091905 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:32.092906 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:32.096100 (+ 3194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:32.096729 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:32.096846 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:32.097442 (+ 596us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:32.097577 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:32.097665 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:32.097787 (+ 122us) server_negotiation.cc:299] Negotiation successful
0618 22:32:32.097864 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:32:32.101902 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 1095
I20260618 22:32:32.103045 1157 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:32.211992 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 1095
I20260618 22:32:32.228755 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 1169
I20260618 22:32:32.229874 1231 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:32.335655 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 1169
I20260618 22:32:32.351336 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 1243
I20260618 22:32:32.352437 1305 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:32.458756 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 1243
2026-06-18T22:32:32Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4288 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:32 dist-test-slave-0956 krb5kdc[1493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:32 dist-test-slave-0956 krb5kdc[1493](info): set up 2 sockets
Jun 18 22:32:32 dist-test-slave-0956 krb5kdc[1493](info): commencing operation
krb5kdc: starting...
W20260618 22:32:33.485641 11211 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.004s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/test-user.keytab.
Jun 18 22:32:33 dist-test-slave-0956 krb5kdc[1493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
Entry for principal kudu/127.10.242.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab.
I20260618 22:32:33.744787 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:33253
--webserver_interface=127.10.242.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:40845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:33253,127.10.242.253:41031,127.10.242.252:42639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:34.071273 1506 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260618 22:32:34.071609 1506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:34.071712 1506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:34.071827 1506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:34.079219 1506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:34.079322 1506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:34.079383 1506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:34.079473 1506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:34.079555 1506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:34.089711 1506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:33253,127.10.242.253:41031,127.10.242.252:42639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.254.keytab
--principal=kudu/127.10.242.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.254:33253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:34.091055 1506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:34.092415 1506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:34.105057 1511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:34.105029 1514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:34.105480 1512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:34.106191 1506 server_base.cc:1034] running on GCE node
I20260618 22:32:34.107033 1506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:34.108513 1506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:34.109756 1506 hybrid_clock.cc:648] HybridClock initialized: now 1781821954109681 us; error 44 us; skew 500 ppm
I20260618 22:32:34.110106 1506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:34 dist-test-slave-0956 krb5kdc[1493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821954, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:34.113268 1506 init.cc:377] Logged in from keytab as kudu/127.10.242.254@KRBTEST.COM (short username kudu)
I20260618 22:32:34.114748 1506 webserver.cc:460] Webserver started at http://127.10.242.254:36899/ using document root <none> and password file <none>
I20260618 22:32:34.115476 1506 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:34.115562 1506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:34.115839 1506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:34.118873 1506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-0/data/instance:
uuid: "09dd224b69d5441595982a17a3f54ea2"
format_stamp: "Formatted at 2026-06-18 22:32:34 on dist-test-slave-0956"
server_key: "881d454b22d4a1584943a329f6ae6b16"
server_key_iv: "b5b034d7987b43652afed793a7106582"
server_key_version: "encryptionkey@0"
I20260618 22:32:34.119632 1506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-0/wal/instance:
uuid: "09dd224b69d5441595982a17a3f54ea2"
format_stamp: "Formatted at 2026-06-18 22:32:34 on dist-test-slave-0956"
server_key: "881d454b22d4a1584943a329f6ae6b16"
server_key_iv: "b5b034d7987b43652afed793a7106582"
server_key_version: "encryptionkey@0"
I20260618 22:32:34.124845 1506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260618 22:32:34.128566 1521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:34.130095 1506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260618 22:32:34.130272 1506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-0/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-0/wal
uuid: "09dd224b69d5441595982a17a3f54ea2"
format_stamp: "Formatted at 2026-06-18 22:32:34 on dist-test-slave-0956"
server_key: "881d454b22d4a1584943a329f6ae6b16"
server_key_iv: "b5b034d7987b43652afed793a7106582"
server_key_version: "encryptionkey@0"
I20260618 22:32:34.130488 1506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:34.147004 1506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:34.151500 1506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:34.151739 1506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:34.169337 1506 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.254:33253
I20260618 22:32:34.169343 1572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.254:33253 every 8 connection(s)
I20260618 22:32:34.170816 1506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:34.174782 1573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:34.179158 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 1506
I20260618 22:32:34.179353 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-0/wal/instance
I20260618 22:32:34.179883 11211 external_mini_cluster.cc:1428] Setting key a2376f6108fe8b7263698903dc84413c
I20260618 22:32:34.182951 1573 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:32:34 dist-test-slave-0956 krb5kdc[1493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821954, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.242.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.253@KRBTEST.COM" created.
I20260618 22:32:34.209235 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.186973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:33253 (local address 127.0.0.1:57310)
0618 22:32:34.190951 (+ 3978us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.191009 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:34.192914 (+ 1905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:34.193894 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:34.193901 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:34.194687 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:34.195053 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:34.195074 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:34.196630 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:34.196634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:34.197780 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:34.197793 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:34.198734 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:34.198741 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:34.198909 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:34.199573 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:34.199598 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:34.201955 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:34.205379 (+ 3424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:34.205390 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:34.205404 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:34.205793 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:34.206193 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:34.206197 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:34.206201 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:34.206350 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:34.207149 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:34.207161 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:34.207374 (+ 213us) client_negotiation.cc:769] Sending connection context
0618 22:32:34.207627 (+ 253us) client_negotiation.cc:240] Negotiation successful
0618 22:32:34.208074 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3737,"thread_start_us":3543,"threads_started":1}
I20260618 22:32:34.209205 1576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.186973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57310 (local address 127.10.242.254:33253)
0618 22:32:34.190474 (+ 3501us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:34.190493 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:34.191155 (+ 662us) server_negotiation.cc:407] Connection header received
0618 22:32:34.193356 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:34.193366 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:34.193655 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:34.193856 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:34.195266 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:34.196477 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:34.197919 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:34.198505 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:34.198689 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:34.202177 (+ 3488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:34.202186 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:34.202200 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:34.202231 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:34.205236 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:34.205934 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:34.205941 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:34.205943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:34.206090 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:34.206465 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:34.206468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:34.206470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:34.207028 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:34.207217 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:34.207872 (+ 655us) server_negotiation.cc:299] Negotiation successful
0618 22:32:34.208270 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":73,"server-negotiator.queue_time_us":3233,"spinlock_wait_cycles":31104,"thread_start_us":2973,"threads_started":1}
I20260618 22:32:34.212515 1573 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:32:34.213636 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.213224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:41031 (local address 127.0.0.1:37262)
0618 22:32:34.213399 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.213517 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260618 22:32:34.213891 1524 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.242.253:41031: connect: Connection refused (error 111)
I20260618 22:32:34.215380 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.215043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:41031 (local address 127.0.0.1:37274)
0618 22:32:34.215188 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.215258 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260618 22:32:34.215628 1573 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.253:41031: Network error: Client connection negotiation failed: client connection to 127.10.242.253:41031: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.253.keytab.
Entry for principal kudu/127.10.242.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.253.keytab.
I20260618 22:32:34.236706 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.253:41031
--webserver_interface=127.10.242.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.253.keytab
--principal=kudu/127.10.242.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:40845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:33253,127.10.242.253:41031,127.10.242.252:42639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:32:34.265010 1573 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } attempt: 1
I20260618 22:32:34.266425 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.265803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:41031 (local address 127.0.0.1:37286)
0618 22:32:34.266052 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.266206 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260618 22:32:34.267544 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.267110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:41031 (local address 127.0.0.1:37300)
0618 22:32:34.267280 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.267377 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260618 22:32:34.267789 1573 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.253:41031: Network error: Client connection negotiation failed: client connection to 127.10.242.253:41031: connect: Connection refused (error 111)
I20260618 22:32:34.336066 1573 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } attempt: 2
I20260618 22:32:34.337392 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.336774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:41031 (local address 127.0.0.1:37308)
0618 22:32:34.337067 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.337216 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260618 22:32:34.338515 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.338085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:41031 (local address 127.0.0.1:37312)
0618 22:32:34.338244 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.338350 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260618 22:32:34.338728 1573 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.253:41031: Network error: Client connection negotiation failed: client connection to 127.10.242.253:41031: connect: Connection refused (error 111)
I20260618 22:32:34.430029 1573 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } attempt: 3
I20260618 22:32:34.431332 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.430774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:41031 (local address 127.0.0.1:37314)
0618 22:32:34.431024 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.431150 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260618 22:32:34.432571 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.432102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:41031 (local address 127.0.0.1:37322)
0618 22:32:34.432285 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.432385 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260618 22:32:34.432818 1573 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.253:41031: Network error: Client connection negotiation failed: client connection to 127.10.242.253:41031: connect: Connection refused (error 111)
W20260618 22:32:34.565248 1580 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260618 22:32:34.565608 1580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:34.565708 1580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:34.565832 1580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:34.573166 1580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:34.573262 1580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:34.573316 1580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:34.573369 1580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:34.573453 1580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:34.576093 1573 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } attempt: 4
I20260618 22:32:34.577373 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.576752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:41031 (local address 127.0.0.1:37338)
0618 22:32:34.577071 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.577208 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260618 22:32:34.578465 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.578075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:41031 (local address 127.0.0.1:37350)
0618 22:32:34.578224 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.578318 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.253:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260618 22:32:34.578715 1573 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.253:41031: Network error: Client connection negotiation failed: client connection to 127.10.242.253:41031: connect: Connection refused (error 111)
I20260618 22:32:34.583994 1580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:33253,127.10.242.253:41031,127.10.242.252:42639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.253.keytab
--principal=kudu/127.10.242.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.253:41031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.242.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:34.585304 1580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:34.586656 1580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:34.597970 1588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:34.598114 1585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:34.598119 1586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:34.598747 1580 server_base.cc:1034] running on GCE node
I20260618 22:32:34.599538 1580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:34.600883 1580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:34.602133 1580 hybrid_clock.cc:648] HybridClock initialized: now 1781821954602051 us; error 46 us; skew 500 ppm
I20260618 22:32:34.602458 1580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:34 dist-test-slave-0956 krb5kdc[1493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821954, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:34.605597 1580 init.cc:377] Logged in from keytab as kudu/127.10.242.253@KRBTEST.COM (short username kudu)
I20260618 22:32:34.607028 1580 webserver.cc:460] Webserver started at http://127.10.242.253:36867/ using document root <none> and password file <none>
I20260618 22:32:34.607702 1580 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:34.607803 1580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:34.608104 1580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:34.611214 1580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-1/data/instance:
uuid: "6f1c61d8b8eb4fd0911a780825f5573d"
format_stamp: "Formatted at 2026-06-18 22:32:34 on dist-test-slave-0956"
server_key: "943196c36c879139c17f23a45842c685"
server_key_iv: "8560b0d5a50b28a08be5c533a3096040"
server_key_version: "encryptionkey@0"
I20260618 22:32:34.612043 1580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-1/wal/instance:
uuid: "6f1c61d8b8eb4fd0911a780825f5573d"
format_stamp: "Formatted at 2026-06-18 22:32:34 on dist-test-slave-0956"
server_key: "943196c36c879139c17f23a45842c685"
server_key_iv: "8560b0d5a50b28a08be5c533a3096040"
server_key_version: "encryptionkey@0"
I20260618 22:32:34.617359 1580 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260618 22:32:34.620752 1595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:34.622239 1580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:32:34.622416 1580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-1/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-1/wal
uuid: "6f1c61d8b8eb4fd0911a780825f5573d"
format_stamp: "Formatted at 2026-06-18 22:32:34 on dist-test-slave-0956"
server_key: "943196c36c879139c17f23a45842c685"
server_key_iv: "8560b0d5a50b28a08be5c533a3096040"
server_key_version: "encryptionkey@0"
I20260618 22:32:34.622628 1580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:34.644488 1580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:34.648825 1580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:34.649099 1580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:34.666383 1580 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.253:41031
I20260618 22:32:34.666401 1646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.253:41031 every 8 connection(s)
I20260618 22:32:34.667748 1580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-1/data/info.pb
I20260618 22:32:34.670763 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 1580
I20260618 22:32:34.670938 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-1/wal/instance
I20260618 22:32:34.671500 11211 external_mini_cluster.cc:1428] Setting key be1bbce946adbb13eb55098e7268ecaf
I20260618 22:32:34.671802 1647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:34.680477 1647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:32:34 dist-test-slave-0956 krb5kdc[1493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821954, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.253@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.242.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.252@KRBTEST.COM" created.
I20260618 22:32:34.701191 1576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.683729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57312 (local address 127.10.242.254:33253)
0618 22:32:34.683896 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:34.683900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:34.685428 (+ 1528us) server_negotiation.cc:407] Connection header received
0618 22:32:34.687322 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:34.687328 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:34.687411 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:34.687497 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:34.689011 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:34.689805 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:34.691289 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:34.691695 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:34.691802 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:34.695547 (+ 3745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:34.695552 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:34.695555 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:34.695589 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:34.698162 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:34.698924 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:34.698930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:34.698934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:34.699075 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:34.699463 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:34.699468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:34.699472 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:34.699766 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:34.699901 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:34.700598 (+ 697us) server_negotiation.cc:299] Negotiation successful
0618 22:32:34.700829 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:32:34.702158 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.684439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:33253 (local address 127.0.0.1:57312)
0618 22:32:34.685266 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.685311 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:34.687049 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:34.687732 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:34.687740 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:34.688267 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:34.688828 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:34.688845 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:34.689960 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:34.689964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:34.691144 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:34.691163 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:34.691909 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:34.691916 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:34.692134 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:34.692840 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:34.692863 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:34.695302 (+ 2439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:34.698293 (+ 2991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:34.698301 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:34.698314 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:34.698803 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:34.699176 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:34.699181 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:34.699185 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:34.699357 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:34.699886 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:34.699893 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:34.700231 (+ 338us) client_negotiation.cc:769] Sending connection context
0618 22:32:34.700506 (+ 275us) client_negotiation.cc:240] Negotiation successful
0618 22:32:34.701015 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":327,"threads_started":1}
I20260618 22:32:34.703830 1647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:32:34 dist-test-slave-0956 krb5kdc[1493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821954, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.253@KRBTEST.COM for kudu/127.10.242.253@KRBTEST.COM
I20260618 22:32:34.717370 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.704566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:41031 (local address 127.0.0.1:37358)
0618 22:32:34.704726 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.704744 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:34.704826 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:34.705783 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:34.705788 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:34.706188 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:34.706320 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:34.706328 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:34.707659 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:34.707663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:34.708555 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:34.708585 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:34.709386 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:34.709391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:34.709564 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:34.710095 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:34.710111 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:34.712007 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:34.714991 (+ 2984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:34.714997 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:34.715001 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:34.715328 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:34.715687 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:34.715692 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:34.715696 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:34.715823 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:34.716365 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:34.716374 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:34.716507 (+ 133us) client_negotiation.cc:769] Sending connection context
0618 22:32:34.716619 (+ 112us) client_negotiation.cc:240] Negotiation successful
0618 22:32:34.716884 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260618 22:32:34.717409 1651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.704736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37358 (local address 127.10.242.253:41031)
0618 22:32:34.705362 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:34.705367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:34.705392 (+ 25us) server_negotiation.cc:407] Connection header received
0618 22:32:34.705433 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:34.705438 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:34.705647 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:34.705774 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:34.706477 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:34.707538 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:34.708686 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:34.709259 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:34.709363 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:34.712190 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:34.712200 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:34.712213 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:34.712240 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:34.714866 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:34.715468 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:34.715474 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:34.715477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:34.715586 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:34.715912 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:34.715916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:34.715918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:34.716247 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:34.716435 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:34.716762 (+ 327us) server_negotiation.cc:299] Negotiation successful
0618 22:32:34.716989 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":484,"thread_start_us":357,"threads_started":1}
I20260618 22:32:34.720002 1647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:32:34.721236 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.720770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:37902)
0618 22:32:34.720933 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.721050 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260618 22:32:34.721556 1597 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
I20260618 22:32:34.722997 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.722672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:37904)
0618 22:32:34.722817 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.722891 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260618 22:32:34.723206 1647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:42639: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.252.keytab.
Entry for principal kudu/127.10.242.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.252.keytab.
I20260618 22:32:34.730146 11211 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
/tmp/dist-test-taskJpXzaW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.252:42639
--webserver_interface=127.10.242.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.252.keytab
--principal=kudu/127.10.242.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.212:40845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.254:33253,127.10.242.253:41031,127.10.242.252:42639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:32:34.772517 1647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } attempt: 1
I20260618 22:32:34.773689 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.773237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:37918)
0618 22:32:34.773420 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.773544 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260618 22:32:34.774664 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.774334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:37922)
0618 22:32:34.774485 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.774559 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260618 22:32:34.774863 1647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:42639: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
I20260618 22:32:34.843070 1647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } attempt: 2
I20260618 22:32:34.844118 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.843694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:37934)
0618 22:32:34.843876 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.843969 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260618 22:32:34.845233 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.844794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:37942)
0618 22:32:34.844975 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.845062 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260618 22:32:34.845523 1647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:42639: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
I20260618 22:32:34.877992 1573 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } attempt: 5
Jun 18 22:32:34 dist-test-slave-0956 krb5kdc[1493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821954, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for kudu/127.10.242.253@KRBTEST.COM
I20260618 22:32:34.895656 1651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.878839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37368 (local address 127.10.242.253:41031)
0618 22:32:34.879030 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:34.879037 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:34.879057 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:32:34.879145 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:34.879151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:34.879229 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:34.879310 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:34.880081 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:34.880777 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:34.881963 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:34.882378 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:34.882485 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:34.890643 (+ 8158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:34.890649 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:34.890652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:34.890687 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:34.893177 (+ 2490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:34.893818 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:34.893824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:34.893827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:34.893944 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:34.894290 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:34.894295 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:34.894299 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:34.894598 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:34.894730 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:34.895107 (+ 377us) server_negotiation.cc:299] Negotiation successful
0618 22:32:34.895332 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:32:34.895654 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.878693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:41031 (local address 127.0.0.1:37368)
0618 22:32:34.878879 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.878901 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:34.879030 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:34.879371 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:34.879375 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:34.879758 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:34.879931 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:34.879940 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:34.880908 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:34.880912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:34.881805 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:34.881818 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:34.882525 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:34.882531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:34.882719 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:34.888128 (+ 5409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:34.888150 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:34.890415 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:34.893314 (+ 2899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:34.893320 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:34.893325 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:34.893702 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:34.894048 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:34.894054 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:34.894058 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:34.894189 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:34.894708 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:34.894713 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:34.894866 (+ 153us) client_negotiation.cc:769] Sending connection context
0618 22:32:34.894996 (+ 130us) client_negotiation.cc:240] Negotiation successful
0618 22:32:34.895289 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260618 22:32:34.896777 1573 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:32:34.897945 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.897578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:37954)
0618 22:32:34.897735 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.897811 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260618 22:32:34.898839 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.898539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:37956)
0618 22:32:34.898670 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.898748 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260618 22:32:34.899015 1573 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:42639: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
I20260618 22:32:34.936796 1647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } attempt: 3
I20260618 22:32:34.937970 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.937530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:37970)
0618 22:32:34.937722 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.937810 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260618 22:32:34.939011 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.938582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:37978)
0618 22:32:34.938806 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.938886 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
W20260618 22:32:34.939246 1647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:42639: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
I20260618 22:32:34.950248 1573 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } attempt: 1
I20260618 22:32:34.951351 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.950909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:37984)
0618 22:32:34.951097 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.951186 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260618 22:32:34.952353 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:34.952037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:38000)
0618 22:32:34.952181 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:34.952248 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260618 22:32:34.952548 1573 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:42639: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
I20260618 22:32:35.020756 1573 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } attempt: 2
I20260618 22:32:35.021816 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.021366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:38012)
0618 22:32:35.021551 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:35.021665 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260618 22:32:35.022817 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.022493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:38028)
0618 22:32:35.022640 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:35.022705 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260618 22:32:35.023000 1573 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:42639: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
W20260618 22:32:35.053793 1654 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260618 22:32:35.054132 1654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:35.054224 1654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:35.054337 1654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:35.061633 1654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:35.061733 1654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:35.061822 1654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:35.061884 1654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:35.061967 1654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:35.071882 1654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.212:40845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.254:33253,127.10.242.253:41031,127.10.242.252:42639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/krb5kdc/kudu_127.10.242.252.keytab
--principal=kudu/127.10.242.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.252:42639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.242.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:35.073213 1654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:35.074488 1654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:32:35.082437 1647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } attempt: 4
I20260618 22:32:35.083452 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.082995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:38030)
0618 22:32:35.083202 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:35.083284 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260618 22:32:35.084535 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.084161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:38038)
0618 22:32:35.084321 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:35.084409 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260618 22:32:35.084820 1647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:42639: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
W20260618 22:32:35.085983 1659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:35.085906 1660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:35.086510 1662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:35.086898 1654 server_base.cc:1034] running on GCE node
I20260618 22:32:35.087661 1654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:35.089110 1654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:35.090363 1654 hybrid_clock.cc:648] HybridClock initialized: now 1781821955090285 us; error 39 us; skew 500 ppm
I20260618 22:32:35.090699 1654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:35 dist-test-slave-0956 krb5kdc[1493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821955, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:35.093935 1654 init.cc:377] Logged in from keytab as kudu/127.10.242.252@KRBTEST.COM (short username kudu)
I20260618 22:32:35.095347 1654 webserver.cc:460] Webserver started at http://127.10.242.252:46315/ using document root <none> and password file <none>
I20260618 22:32:35.096068 1654 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:35.096168 1654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:35.096490 1654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:35.099505 1654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-2/data/instance:
uuid: "762e5c8ca4b44d668e75079570fdae8e"
format_stamp: "Formatted at 2026-06-18 22:32:35 on dist-test-slave-0956"
server_key: "f5651f344ac7463be36b916f6550bd4c"
server_key_iv: "a18be16c98ca50f645a0d025a9d9708e"
server_key_version: "encryptionkey@0"
I20260618 22:32:35.100371 1654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-2/wal/instance:
uuid: "762e5c8ca4b44d668e75079570fdae8e"
format_stamp: "Formatted at 2026-06-18 22:32:35 on dist-test-slave-0956"
server_key: "f5651f344ac7463be36b916f6550bd4c"
server_key_iv: "a18be16c98ca50f645a0d025a9d9708e"
server_key_version: "encryptionkey@0"
I20260618 22:32:35.106910 1654 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260618 22:32:35.110918 1669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:35.112537 1654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:32:35.112715 1654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-2/data,/tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-2/wal
uuid: "762e5c8ca4b44d668e75079570fdae8e"
format_stamp: "Formatted at 2026-06-18 22:32:35 on dist-test-slave-0956"
server_key: "f5651f344ac7463be36b916f6550bd4c"
server_key_iv: "a18be16c98ca50f645a0d025a9d9708e"
server_key_version: "encryptionkey@0"
I20260618 22:32:35.112968 1654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:35.129189 1573 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } attempt: 3
I20260618 22:32:35.130249 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.129769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:38054)
0618 22:32:35.130010 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:35.130094 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260618 22:32:35.131302 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.130933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:38068)
0618 22:32:35.131128 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:35.131197 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260618 22:32:35.131306 1654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
W20260618 22:32:35.131512 1573 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.252:42639: Network error: Client connection negotiation failed: client connection to 127.10.242.252:42639: connect: Connection refused (error 111)
I20260618 22:32:35.135777 1654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:35.136008 1654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:35.153378 1654 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.252:42639
I20260618 22:32:35.153411 1720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.252:42639 every 8 connection(s)
I20260618 22:32:35.154762 1654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-2/data/info.pb
I20260618 22:32:35.158541 1721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:35.163959 11211 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu as pid 1654
I20260618 22:32:35.164129 11211 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJpXzaW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821604371818-11211-0/minicluster-data/master-2/wal/instance
I20260618 22:32:35.164597 11211 external_mini_cluster.cc:1428] Setting key df4f351e60ed6c11c941bb454f7a9766
I20260618 22:32:35.165220 1721 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:32:35 dist-test-slave-0956 krb5kdc[1493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821955, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.252@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
Jun 18 22:32:35 dist-test-slave-0956 krb5kdc[1493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.252@KRBTEST.COM
I20260618 22:32:35.187350 1576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.168178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57324 (local address 127.10.242.254:33253)
0618 22:32:35.168344 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.168348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.172235 (+ 3887us) server_negotiation.cc:407] Connection header received
0618 22:32:35.172548 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.172552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.172618 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.172700 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.174003 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.174716 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.176803 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.177329 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.177459 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:35.181318 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:35.181326 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:35.181331 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:35.181369 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:35.184135 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.184989 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.184993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.184996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.185178 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.185604 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.185610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.185614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.185887 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:35.186022 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:35.186764 (+ 742us) server_negotiation.cc:299] Negotiation successful
0618 22:32:35.186996 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260618 22:32:35.188439 1725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.168210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.254:33253 (local address 127.0.0.1:57324)
0618 22:32:35.172098 (+ 3888us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:35.172139 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:35.172356 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:35.172784 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:35.172791 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:35.173504 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:35.173831 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:35.173844 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.174883 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.174888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:35.176670 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:35.176682 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.177573 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.177580 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:35.177754 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:35.178470 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:35.178502 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:35.181064 (+ 2562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:35.184277 (+ 3213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:35.184299 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:35.184313 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:35.184856 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:35.185282 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:35.185295 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:35.185299 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:35.185487 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:35.186013 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:35.186021 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:35.186386 (+ 365us) client_negotiation.cc:769] Sending connection context
0618 22:32:35.186643 (+ 257us) client_negotiation.cc:240] Negotiation successful
0618 22:32:35.187178 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3643,"mutex_wait_us":5,"thread_start_us":3494,"threads_started":1}
I20260618 22:32:35.188457 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.166416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38072 (local address 127.10.242.252:42639)
0618 22:32:35.167171 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.167182 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.167223 (+ 41us) server_negotiation.cc:407] Connection header received
0618 22:32:35.169930 (+ 2707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.169964 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.170571 (+ 607us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.171054 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.172834 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.174737 (+ 1903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.176588 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.177484 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.177723 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:35.181887 (+ 4164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:35.181897 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:35.181911 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:35.181944 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:35.184744 (+ 2800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.185639 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.185645 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.185648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.185781 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.186147 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.186151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.186153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.186537 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:35.186818 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:35.187127 (+ 309us) server_negotiation.cc:299] Negotiation successful
0618 22:32:35.187358 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":550,"thread_start_us":327,"threads_started":1}
I20260618 22:32:35.190018 1721 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:32:35 dist-test-slave-0956 krb5kdc[1493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821955, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.252@KRBTEST.COM for kudu/127.10.242.253@KRBTEST.COM
Jun 18 22:32:35 dist-test-slave-0956 krb5kdc[1493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.254@KRBTEST.COM
I20260618 22:32:35.204568 1725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.190693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.253:41031 (local address 127.0.0.1:37380)
0618 22:32:35.190849 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:35.190866 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:35.190962 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:35.191263 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:35.191339 (+ 76us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:35.191710 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:35.191904 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:35.191915 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.193323 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.193330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:35.194402 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:35.194416 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.195188 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.195195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:35.195421 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:35.196015 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:35.196034 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:35.198146 (+ 2112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:35.201903 (+ 3757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:35.201908 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:35.201913 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:35.202324 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:35.202783 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:35.202787 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:35.202789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:35.202954 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:35.203618 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:35.203623 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:35.203828 (+ 205us) client_negotiation.cc:769] Sending connection context
0618 22:32:35.203999 (+ 171us) client_negotiation.cc:240] Negotiation successful
0618 22:32:35.204308 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260618 22:32:35.204666 1651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.190782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37380 (local address 127.10.242.253:41031)
0618 22:32:35.190951 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.190956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.190974 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:32:35.191061 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.191067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.191133 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.191213 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.192065 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.193162 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.194580 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.195038 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.195164 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:35.198355 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:35.198361 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:35.198364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:35.198400 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:35.201702 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.202459 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.202465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.202469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.202636 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.203068 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.203073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.203077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.203464 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:35.203628 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:35.204121 (+ 493us) server_negotiation.cc:299] Negotiation successful
0618 22:32:35.204311 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260618 22:32:35.205925 1721 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:32:35.206400 1576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.191699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57336 (local address 127.10.242.254:33253)
0618 22:32:35.191878 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.191882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.191900 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:32:35.192005 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.192010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.192081 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.192149 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.193657 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.194400 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.195554 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.196106 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.196216 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:35.200075 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:35.200083 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:35.200087 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:35.200133 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:35.203526 (+ 3393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.204426 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.204432 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.204436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.204586 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.205044 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.205050 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.205053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.205394 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:35.205677 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:35.205949 (+ 272us) server_negotiation.cc:299] Negotiation successful
0618 22:32:35.206141 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
Jun 18 22:32:35 dist-test-slave-0956 krb5kdc[1493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821955, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.252@KRBTEST.COM for kudu/127.10.242.252@KRBTEST.COM
Jun 18 22:32:35 dist-test-slave-0956 krb5kdc[1493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.253@KRBTEST.COM
I20260618 22:32:35.220026 1725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.206804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:38078)
0618 22:32:35.207019 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:35.207038 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:35.207148 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:35.207486 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:35.207491 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:35.207987 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:35.208167 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:35.208179 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.209124 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.209130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:35.210350 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:35.210362 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.211251 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.211256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:35.211494 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:35.212086 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:35.212108 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:35.214597 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:35.217537 (+ 2940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:35.217542 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:35.217545 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:35.217931 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:35.218327 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:35.218331 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:35.218335 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:35.218554 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:35.219161 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:35.219164 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:35.219342 (+ 178us) client_negotiation.cc:769] Sending connection context
0618 22:32:35.219527 (+ 185us) client_negotiation.cc:240] Negotiation successful
0618 22:32:35.219748 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260618 22:32:35.220275 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.207106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38078 (local address 127.10.242.252:42639)
0618 22:32:35.207243 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.207247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.207260 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:32:35.207303 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.207307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.207358 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.207442 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.208314 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.208965 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.210480 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.211103 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.211216 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:35.214812 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:35.214817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:35.214821 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:35.214850 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:35.217434 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.218052 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.218057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.218061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.218207 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.218651 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.218656 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.218660 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.219042 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:35.219187 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:35.219657 (+ 470us) server_negotiation.cc:299] Negotiation successful
0618 22:32:35.219939 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":42}
I20260618 22:32:35.223326 1651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.208148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37394 (local address 127.10.242.253:41031)
0618 22:32:35.208380 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.208386 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.208402 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:32:35.208478 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.208484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.208542 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.208615 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.210094 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.211051 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.212776 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.213301 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.213411 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:35.217734 (+ 4323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:35.217739 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:35.217742 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:35.217778 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:35.220767 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.221556 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.221561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.221565 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.221696 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.222053 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.222056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.222058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.222311 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:35.222562 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:35.222835 (+ 273us) server_negotiation.cc:299] Negotiation successful
0618 22:32:35.223042 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260618 22:32:35.225281 1721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e: Bootstrap starting.
I20260618 22:32:35.228173 11211 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:32:35.228706 1721 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:35.229769 1721 log.cc:824] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:35.232259 1721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e: No bootstrap required, opened a new log
I20260618 22:32:35.230504 1548 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "09dd224b69d5441595982a17a3f54ea2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57336
I20260618 22:32:35.242527 1721 raft_consensus.cc:348] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } } peers { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } } peers { permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } }
I20260618 22:32:35.242970 1721 raft_consensus.cc:729] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 762e5c8ca4b44d668e75079570fdae8e, State: Initialized, Role: FOLLOWER
I20260618 22:32:35.243446 1721 consensus_queue.cc:260] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } } peers { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } } peers { permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } }
I20260618 22:32:35.244370 1727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } } peers { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } } peers { permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } } }
I20260618 22:32:35.244695 1727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:32:35.245601 1721 sys_catalog.cc:564] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e [sys.catalog]: configured and running, proceeding with master startup.
W20260618 22:32:35.255182 1738 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:32:35.255336 1738 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:32:35.252539 1654 master_runner.cc:186] Error getting master registration for 127.10.242.254:33253: OK, instance_id { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" instance_seqno: 1781821954164200 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 22:32:35.263770 1654 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260618 22:32:35.308713 1573 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } attempt: 4
Jun 18 22:32:35 dist-test-slave-0956 krb5kdc[1493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821954, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.254@KRBTEST.COM for kudu/127.10.242.252@KRBTEST.COM
I20260618 22:32:35.324355 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.309634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:38088)
0618 22:32:35.309836 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:35.309863 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:35.310040 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:35.310387 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:35.310392 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:35.310787 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:35.310956 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:35.310965 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.312408 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.312414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:35.313470 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:35.313487 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.314219 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.314225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:35.314424 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:35.314966 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:35.314983 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:35.317441 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:35.321519 (+ 4078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:35.321525 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:35.321529 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:35.322069 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:35.322433 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:35.322439 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:35.322443 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:35.322614 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:35.323341 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:35.323346 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:35.323517 (+ 171us) client_negotiation.cc:769] Sending connection context
0618 22:32:35.323667 (+ 150us) client_negotiation.cc:240] Negotiation successful
0618 22:32:35.323958 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260618 22:32:35.324363 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.309663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38088 (local address 127.10.242.252:42639)
0618 22:32:35.309837 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.309843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.309949 (+ 106us) server_negotiation.cc:407] Connection header received
0618 22:32:35.310165 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.310171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.310256 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.310362 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.311105 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.312270 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.313611 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.314077 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.314181 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:35.317620 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:35.317627 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:35.317632 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:35.317672 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:35.321375 (+ 3703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.322183 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.322188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.322192 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.322331 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.322717 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.322722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.322725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.323207 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:35.323348 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:35.323760 (+ 412us) server_negotiation.cc:299] Negotiation successful
0618 22:32:35.323996 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:32:35.330176 1573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2: Bootstrap starting.
I20260618 22:32:35.334137 1573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:35.335397 1573 log.cc:824] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:35.338905 1573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2: No bootstrap required, opened a new log
I20260618 22:32:35.346660 1573 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } } peers { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } } peers { permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } }
I20260618 22:32:35.347096 1573 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09dd224b69d5441595982a17a3f54ea2, State: Initialized, Role: FOLLOWER
I20260618 22:32:35.347631 1573 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } } peers { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } } peers { permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } }
I20260618 22:32:35.348675 1741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } } peers { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } } peers { permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } } }
I20260618 22:32:35.349045 1741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:32:35.350014 1573 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:35.350071 1548 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260618 22:32:35.350255 1548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:35.352923 1548 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } } peers { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } } peers { permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } }
I20260618 22:32:35.355213 1696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09dd224b69d5441595982a17a3f54ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "762e5c8ca4b44d668e75079570fdae8e"
I20260618 22:32:35.355618 1696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:35.358007 1696 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09dd224b69d5441595982a17a3f54ea2 in term 1.
I20260618 22:32:35.358464 1548 leader_election.cc:288] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [CANDIDATE]: Term 1 election: Requested vote from peers 6f1c61d8b8eb4fd0911a780825f5573d (127.10.242.253:41031), 762e5c8ca4b44d668e75079570fdae8e (127.10.242.252:42639)
I20260618 22:32:35.358696 1523 leader_election.cc:302] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09dd224b69d5441595982a17a3f54ea2, 762e5c8ca4b44d668e75079570fdae8e; no voters:
I20260618 22:32:35.359148 1741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:35.374505 1741 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [term 1 LEADER]: Becoming Leader. State: Replica: 09dd224b69d5441595982a17a3f54ea2, State: Running, Role: LEADER
I20260618 22:32:35.375224 1741 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } } peers { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } } peers { permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } }
I20260618 22:32:35.384037 1647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } attempt: 5
W20260618 22:32:35.388265 1675 tablet.cc:2369] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:32:35.388173 1748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09dd224b69d5441595982a17a3f54ea2. Latest consensus state: current_term: 1 leader_uuid: "09dd224b69d5441595982a17a3f54ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } } peers { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } } peers { permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } } }
I20260618 22:32:35.388563 1748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:35.394464 1762 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260618 22:32:35.404659 1527 tablet.cc:2369] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 18 22:32:35 dist-test-slave-0956 krb5kdc[1493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821954, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.253@KRBTEST.COM for kudu/127.10.242.252@KRBTEST.COM
I20260618 22:32:35.413100 1762 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:35.425128 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.384654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.252:42639 (local address 127.0.0.1:38112)
0618 22:32:35.385494 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:35.385517 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:35.385656 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:35.395003 (+ 9347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:35.395008 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:35.395503 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:35.395716 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:35.395731 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.398507 (+ 2776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.398514 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:35.399975 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:35.399994 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.401672 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.401678 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:35.401922 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:35.402669 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:35.402693 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:35.405975 (+ 3282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:35.416953 (+ 10978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:35.416959 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:35.416962 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:35.417494 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:35.419459 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:35.419465 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:35.419468 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:35.419702 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:35.424047 (+ 4345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:35.424053 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:35.424248 (+ 195us) client_negotiation.cc:769] Sending connection context
0618 22:32:35.424400 (+ 152us) client_negotiation.cc:240] Negotiation successful
0618 22:32:35.424726 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718}
I20260618 22:32:35.429891 1761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.386193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38112 (local address 127.10.242.252:42639)
0618 22:32:35.394582 (+ 8389us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.394588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.394617 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:32:35.394692 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.394698 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.394790 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.394940 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.396045 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.398252 (+ 2207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.400246 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.401480 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.402344 (+ 864us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:35.406537 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:35.406546 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:35.406551 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:35.406625 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:35.416513 (+ 9888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.417981 (+ 1468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.417988 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.418147 (+ 159us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.418981 (+ 834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.420187 (+ 1206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.420193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.420197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.423893 (+ 3696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:35.425387 (+ 1494us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:35.425521 (+ 134us) server_negotiation.cc:299] Negotiation successful
0618 22:32:35.429412 (+ 3891us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8248,"thread_start_us":8033,"threads_started":1}
I20260618 22:32:35.432053 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.386078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38096 (local address 127.10.242.252:42639)
0618 22:32:35.386630 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.386637 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.400927 (+ 14290us) server_negotiation.cc:407] Connection header received
0618 22:32:35.401469 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.401475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.401571 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.401639 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.406267 (+ 4628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.407337 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.414124 (+ 6787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.414781 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.414882 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:35.421374 (+ 6492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:35.421381 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:35.421385 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:35.421421 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:35.425908 (+ 4487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.428661 (+ 2753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.428667 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.428671 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.428824 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.430285 (+ 1461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.430292 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.430296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.430654 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:35.430901 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:35.431460 (+ 559us) server_negotiation.cc:299] Negotiation successful
0618 22:32:35.431718 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434}
I20260618 22:32:35.433758 1576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.384926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57342 (local address 127.10.242.254:33253)
0618 22:32:35.385904 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.385911 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.400968 (+ 15057us) server_negotiation.cc:407] Connection header received
0618 22:32:35.401478 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.401483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.401567 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.401633 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.408179 (+ 6546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.409295 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.414049 (+ 4754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.414893 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.415023 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:35.421678 (+ 6655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:35.421686 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:35.421690 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:35.421753 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:35.428409 (+ 6656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.429248 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.429258 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.429261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.429437 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.429878 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.429883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.429886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.430239 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:35.430494 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:35.433076 (+ 2582us) server_negotiation.cc:299] Negotiation successful
0618 22:32:35.433372 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":832}
I20260618 22:32:35.440552 1647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d: Bootstrap starting.
I20260618 22:32:35.443653 1651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:35.385311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37406 (local address 127.10.242.253:41031)
0618 22:32:35.386489 (+ 1178us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.386496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.406357 (+ 19861us) server_negotiation.cc:407] Connection header received
0618 22:32:35.406794 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.406800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.406875 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.406937 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.413364 (+ 6427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.414467 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.426436 (+ 11969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:35.427128 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:35.427225 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:35.436044 (+ 8819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:35.436054 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:35.436060 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:35.436105 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:35.439915 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.441010 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.441017 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.441020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.441217 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:35.441675 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:35.441680 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:35.441683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:35.442078 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:35.442401 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:35.442978 (+ 577us) server_negotiation.cc:299] Negotiation successful
0618 22:32:35.443271 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1063}
I20260618 22:32:35.444715 1647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:35.445540 1764 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260618 22:32:35.446132 1647 log.cc:824] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:35.454566 1696 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e [term 1 FOLLOWER]: Refusing update from remote peer 09dd224b69d5441595982a17a3f54ea2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:32:35.455596 1748 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:35.451503 1506 master_runner.cc:186] Error getting master registration for 127.10.242.253:41031: OK, instance_id { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" instance_seqno: 1781821954661576 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 22:32:35.479512 1506 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260618 22:32:35.480155 1727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09dd224b69d5441595982a17a3f54ea2. Latest consensus state: current_term: 1 leader_uuid: "09dd224b69d5441595982a17a3f54ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } } peers { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } } peers { permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } } }
I20260618 22:32:35.480510 1727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:32:35.482959 1647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d: No bootstrap required, opened a new log
I20260618 22:32:35.484704 1765 mvcc.cc:205] Tried to move back new op lower bound from 7298342729526657024 to 7298342729223233536. Current Snapshot: MvccSnapshot[applied={T|T < 7298342729526657024}]
I20260618 22:32:35.488960 1727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "09dd224b69d5441595982a17a3f54ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } } peers { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } } peers { permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } } }
I20260618 22:32:35.489270 1727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:32:35.495245 1767 mvcc.cc:205] Tried to move back new op lower bound from 7298342729526657024 to 7298342729223233536. Current Snapshot: MvccSnapshot[applied={T|T < 7298342729526657024}]
I20260618 22:32:35.500008 1741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "09dd224b69d5441595982a17a3f54ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } } peers { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } } peers { permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } } }
I20260618 22:32:35.500337 1741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:35.502597 1647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } } peers { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } } peers { permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } }
I20260618 22:32:35.503114 1647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f1c61d8b8eb4fd0911a780825f5573d, State: Initialized, Role: FOLLOWER
I20260618 22:32:35.503679 1647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } } peers { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } } peers { permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } }
I20260618 22:32:35.505180 1762 catalog_manager.cc:1347] Generated new cluster ID: f5a2868458a4406a8db36d68eadad3e5
I20260618 22:32:35.505290 1762 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:35.509760 1771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } } peers { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } } peers { permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } } }
I20260618 22:32:35.510542 1771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:32:35.511171 1647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:35.511237 1622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09dd224b69d5441595982a17a3f54ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6f1c61d8b8eb4fd0911a780825f5573d"
I20260618 22:32:35.511659 1622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:35.515029 1622 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09dd224b69d5441595982a17a3f54ea2 in term 1.
I20260618 22:32:35.515843 1621 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d [term 1 FOLLOWER]: Refusing update from remote peer 09dd224b69d5441595982a17a3f54ea2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:32:35.517506 1748 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:35.528559 1762 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:35.530457 1762 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:35.552614 1762 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2: Generated new TSK 0
I20260618 22:32:35.553790 1762 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:32:35.597587 1771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09dd224b69d5441595982a17a3f54ea2. Latest consensus state: current_term: 1 leader_uuid: "09dd224b69d5441595982a17a3f54ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } } peers { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } } peers { permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } } }
I20260618 22:32:35.597939 1771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:32:35.598464 1781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "09dd224b69d5441595982a17a3f54ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } } peers { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } } peers { permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } } }
I20260618 22:32:35.600078 1741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "09dd224b69d5441595982a17a3f54ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dd224b69d5441595982a17a3f54ea2" member_type: VOTER last_known_addr { host: "127.10.242.254" port: 33253 } } peers { permanent_uuid: "6f1c61d8b8eb4fd0911a780825f5573d" member_type: VOTER last_known_addr { host: "127.10.242.253" port: 41031 } } peers { permanent_uuid: "762e5c8ca4b44d668e75079570fdae8e" member_type: VOTER last_known_addr { host: "127.10.242.252" port: 42639 } } }
I20260618 22:32:35.600370 1741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09dd224b69d5441595982a17a3f54ea2 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:35.602485 1781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:32:35.632540 1789 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260618 22:32:35.663916 1651 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:35.650539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37422 (local address 127.10.242.253:41031)
0618 22:32:35.650747 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.650753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.661770 (+ 11017us) server_negotiation.cc:407] Connection header received
0618 22:32:35.662086 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.662093 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.662192 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.662267 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.663618 (+ 1351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37422: BlockingRecv error: recv got EOF from 127.0.0.1:37422 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260618 22:32:35.666427 1576 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:35.649903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57356 (local address 127.10.242.254:33253)
0618 22:32:35.650387 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.650394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.664813 (+ 14419us) server_negotiation.cc:407] Connection header received
0618 22:32:35.665100 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.665106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.665204 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.665278 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.666195 (+ 917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57356: BlockingRecv error: recv got EOF from 127.0.0.1:57356 (error 108)
Metrics: {"server-negotiator.queue_time_us":306}
W20260618 22:32:35.667949 1761 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:35.655506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38116 (local address 127.10.242.252:42639)
0618 22:32:35.656204 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.656211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.665770 (+ 9559us) server_negotiation.cc:407] Connection header received
0618 22:32:35.666495 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.666502 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.666610 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.666686 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.667714 (+ 1028us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38116: BlockingRecv error: recv got EOF from 127.0.0.1:38116 (error 108)
Metrics: {"server-negotiator.queue_time_us":544}
W20260618 22:32:35.698014 1651 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:35.695692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37434 (local address 127.10.242.253:41031)
0618 22:32:35.695871 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.695878 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.696369 (+ 491us) server_negotiation.cc:407] Connection header received
0618 22:32:35.696632 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.696638 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.696773 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.696894 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.697750 (+ 856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37434: BlockingRecv error: recv got EOF from 127.0.0.1:37434 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260618 22:32:35.711354 1651 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:35.706839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37446 (local address 127.10.242.253:41031)
0618 22:32:35.707027 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.707033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.709651 (+ 2618us) server_negotiation.cc:407] Connection header received
0618 22:32:35.709874 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.709880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.709991 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.710094 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.711131 (+ 1037us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37446: BlockingRecv error: recv got EOF from 127.0.0.1:37446 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260618 22:32:35.727377 1651 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:35.722614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37448 (local address 127.10.242.253:41031)
0618 22:32:35.722801 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.722808 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.725650 (+ 2842us) server_negotiation.cc:407] Connection header received
0618 22:32:35.725938 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.725945 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.726061 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.726181 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.727127 (+ 946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37448: BlockingRecv error: recv got EOF from 127.0.0.1:37448 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260618 22:32:35.741298 1651 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:35.739204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37460 (local address 127.10.242.253:41031)
0618 22:32:35.739439 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.739445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.739772 (+ 327us) server_negotiation.cc:407] Connection header received
0618 22:32:35.739990 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.739995 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.740108 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.740216 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.741021 (+ 805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37460: BlockingRecv error: recv got EOF from 127.0.0.1:37460 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260618 22:32:35.762187 1651 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:35.759977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.10.242.253:41031)
0618 22:32:35.760151 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.760157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.760678 (+ 521us) server_negotiation.cc:407] Connection header received
0618 22:32:35.760874 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.760880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.760985 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.761122 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.761951 (+ 829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55712: BlockingRecv error: recv got EOF from 127.0.0.1:55712 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260618 22:32:35.783820 1651 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:35.781826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55716 (local address 127.10.242.253:41031)
0618 22:32:35.782027 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.782032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.782355 (+ 323us) server_negotiation.cc:407] Connection header received
0618 22:32:35.782584 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.782588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.782675 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.782762 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.783590 (+ 828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55716: BlockingRecv error: recv got EOF from 127.0.0.1:55716 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260618 22:32:35.821830 1651 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:35.819913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55732 (local address 127.10.242.253:41031)
0618 22:32:35.820129 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.820135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.820438 (+ 303us) server_negotiation.cc:407] Connection header received
0618 22:32:35.820614 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.820620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.820724 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.820848 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.821573 (+ 725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55732: BlockingRecv error: recv got EOF from 127.0.0.1:55732 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260618 22:32:35.897045 1651 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:35.895053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55738 (local address 127.10.242.253:41031)
0618 22:32:35.895249 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:35.895256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:35.895637 (+ 381us) server_negotiation.cc:407] Connection header received
0618 22:32:35.895802 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:35.895808 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:35.895907 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:35.896036 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:35.896801 (+ 765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55738: BlockingRecv error: recv got EOF from 127.0.0.1:55738 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260618 22:32:36.030037 1651 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:36.028185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55750 (local address 127.10.242.253:41031)
0618 22:32:36.028391 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:36.028399 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:36.028704 (+ 305us) server_negotiation.cc:407] Connection header received
0618 22:32:36.028900 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:36.028905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:36.028980 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:36.029066 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:36.029845 (+ 779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55750: BlockingRecv error: recv got EOF from 127.0.0.1:55750 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:32:36.257283 1738 catalog_manager.cc:1259] Loaded cluster ID: f5a2868458a4406a8db36d68eadad3e5
I20260618 22:32:36.257385 1738 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e: loading cluster ID for follower catalog manager: success
I20260618 22:32:36.259806 1738 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e: acquiring CA information for follower catalog manager: success
I20260618 22:32:36.261569 1738 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 762e5c8ca4b44d668e75079570fdae8e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260618 22:32:36.291291 1651 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:32:36.289710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55752 (local address 127.10.242.253:41031)
0618 22:32:36.289922 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:36.289928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:36.290042 (+ 114us) server_negotiation.cc:407] Connection header received
0618 22:32:36.290260 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:36.290265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:36.290346 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:36.290430 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:36.291128 (+ 698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55752: BlockingRecv error: recv got EOF from 127.0.0.1:55752 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:32:36.636821 1789 catalog_manager.cc:1259] Loaded cluster ID: f5a2868458a4406a8db36d68eadad3e5
I20260618 22:32:36.636950 1789 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d: loading cluster ID for follower catalog manager: success
I20260618 22:32:36.639628 1789 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d: acquiring CA information for follower catalog manager: success
I20260618 22:32:36.641716 1789 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6f1c61d8b8eb4fd0911a780825f5573d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 22:32:36.813130 1853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:36.807032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55760 (local address 127.10.242.253:41031)
0618 22:32:36.807811 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:36.807817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:36.807843 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:32:36.807903 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:36.807907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:36.808062 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:36.808170 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:36.808604 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:36.809435 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:36.810739 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:36.811184 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:36.811322 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:36.811813 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:36.812466 (+ 653us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:36.812609 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:36.812720 (+ 111us) server_negotiation.cc:299] Negotiation successful
0618 22:32:36.812811 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":488,"threads_started":1}
I20260618 22:32:36.830519 1859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:36.819766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36674 (local address 127.10.242.252:42639)
0618 22:32:36.825500 (+ 5734us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:36.825505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:36.825526 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:32:36.825621 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:36.825626 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:36.825765 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:36.825936 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:36.826363 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:36.827139 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:36.828428 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:36.828820 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:36.828927 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:36.829456 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:36.829953 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:36.830111 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:36.830160 (+ 49us) server_negotiation.cc:299] Negotiation successful
0618 22:32:36.830241 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5616,"thread_start_us":5489,"threads_started":1}
I20260618 22:32:36.857846 1853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:36.837375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.10.242.253:41031)
0618 22:32:36.837541 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:36.837548 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:36.847896 (+ 10348us) server_negotiation.cc:407] Connection header received
0618 22:32:36.848878 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:36.848915 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:36.849049 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:36.849187 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:36.851009 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:36.852717 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:36.855604 (+ 2887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:36.856227 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:36.856341 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:36.856980 (+ 639us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:36.857132 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:36.857195 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:36.857455 (+ 260us) server_negotiation.cc:299] Negotiation successful
0618 22:32:36.857538 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260618 22:32:36.859005 1859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:36.837745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36688 (local address 127.10.242.252:42639)
0618 22:32:36.837919 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:36.837924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:36.850807 (+ 12883us) server_negotiation.cc:407] Connection header received
0618 22:32:36.851224 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:36.851229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:36.851352 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:36.851414 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:36.852132 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:36.853191 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:36.856644 (+ 3453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:36.857260 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:36.857352 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:36.857972 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:36.858117 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:36.858178 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:36.858642 (+ 464us) server_negotiation.cc:299] Negotiation successful
0618 22:32:36.858732 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:32:36.861289 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:36.837047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37262 (local address 127.10.242.254:33253)
0618 22:32:36.850239 (+ 13192us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:36.850245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:36.854432 (+ 4187us) server_negotiation.cc:407] Connection header received
0618 22:32:36.854559 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:36.854564 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:36.854794 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:36.855025 (+ 231us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:36.855830 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:36.856836 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:36.858723 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:36.859380 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:36.859487 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:36.860218 (+ 731us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:36.860672 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:36.860765 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:36.860901 (+ 136us) server_negotiation.cc:299] Negotiation successful
0618 22:32:36.860987 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13059,"thread_start_us":13754,"threads_started":1}
I20260618 22:32:36.865832 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 1506
I20260618 22:32:36.867864 1568 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:36.978720 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 1506
W20260618 22:32:37.000392 1598 connection.cc:535] server connection from 127.0.0.1:37368 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260618 22:32:37.000885 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 1580
I20260618 22:32:37.002097 1642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:37.107167 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 1580
I20260618 22:32:37.124382 11211 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu pid 1654
I20260618 22:32:37.125484 1716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:37.231375 11211 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJpXzaW/build/asan/bin/kudu with pid 1654
2026-06-18T22:32:37Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4773 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9062 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (352819 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST